131–140 of 161 posts
This is a screenshot of my google search from 2 days ago: http://i.imgur.com/BNAcSsn.png I've been using Lambda quite a bit, I think it's SO amazingly useful. Tasks that are highly parallelized and CPU intensive can literally be infinitely scaled out. I find it weird that their poster child use case is still always a reactive event like watching S3 and formatting images. There are so many use cases for directly invok…
Earlier quoted context omitted.
With AWS API gateway + lambda + dynamodb, I maintain exactly zero servers. Zero physical servers. Zero server OSes. Zero sysadmin. It's perfectly reasonable IMO for that to be called "serverless".
With my car dealer, I maintain exactly zero factories. Zero physical factories. Zero factory workers. It's perfectly reasonable IMO for that to be called "factory-less car".
I wonder if Amazon will ever open Lambda up to any Docker image? (I know it's possible to run binaries, but its a bit of a pain to compile with the Amazon AMI, etc.) Being able to have a bunch of `docker run` with any image would be pretty powerful.
I've always had one big question about Lambda. Is it really worth the cost you get for the convienience of it? Is anyone using it in production that can comment?
I think you can do the math yourself - the costs are published. FYI - we did some experiments and the limiting factor was latency. 250-300ms on average, you have to go through their API feature as well, and that's part of the delay. But worse - Lamda's that have not been called for several minutes (I'm assuming they are not 'hot') often take several seconds, up to 5s to be called. So it creates a problem for intermit…
OT: I teach computational methods and even as much as I dislike teaching/conflating it with web dev, I have included "let's build a web app" because students like building and deploying a thing, and because Heroku has a free tier. I've considered the possibility of having students do things on AWS (beyond web dev), including Lambda, and just expensing the costs. It seems feasible to quickly set up every student with…
I would just set up a new account for each student, have them use their own billing info, have them use the free tier, teach them how to set up billing alerts, and let them go to town. They're going to need to learn to take cost into account when working at a real job with AWS so this is the best way to teach them to take accountability.
Earlier quoted context omitted.
By comparison (since I was curious), 70 hours on an m3.medium spot instance will run around $0.70. On an on-demand, it's about $5.39. EMR will cost you about $7.00 on top of the EC2 costs. If you can peg the CPU and don't mind getting interrupted, spot instances are still a fair bit cheaper. But Lambda looks pretty attractive for any other use-case where the statelessness of Lambda doesn't bite you.
Keep in mind that with EC2 you're billed hourly so the fastest a 70 cpu-hour job could finish on m3.medium for $0.70 is 1 hour, and that's ignoring setup time, etc. Meanwhile, on Lambda, you can actually run 1600 60s jobs (or 27 CPU-hours) in 3 minutes. This is inclusive of setup time, job submission, stragglers, etc. [1] Of course, if you've got sustained load, it's cheaper to go with spot instances, but the "occasi…
Joyent Manta: https://www.joyent.com/manta
Hyper: http://hyper.sh
Possibly Joyent Triton: https://www.joyent.com/triton
I personally often want to run a bunch of things for ~1-15 minutes, and have too much data or setup to fit neatly in a lambda function. However, I don't need 1000 things running simultaneously, although manta would help still there.
I'd love to see some more layers over the top of services like this, hopefully someday getting us back to picloud. I miss that service.
Earlier quoted context omitted.
Check out Hyper. https://www.hyper.sh/ "With HyperContainer, we make the performance/overhead of virtualized container similar to linux container --- 130ms launch time, and sharing the read-only part of memory (kernel and init)." -gnawux https://news.ycombinator.com/item?id=12873089
Yep, and "hyper func" in the roadmap
Earlier quoted context omitted.
So google compute setup I did a while back with preemptible instances + a celery queue + some autoscale based on load... The guts to make all that work was 50 or so lines of config. I think my auto scale script was 20 lines or so of Python. I guess the biggest downside was spinning up the new server took about 2 minutes, so for big load spikes it took a bit for it to level out... but with GCE per minute billing, all…
That's essentially what Google Cloud Dataproc gives you (managed Hadoop/Spark): - Per-minute billing - 0-to-cluster in under 90 seconds (aim for 30 seconds) - Pre-emptiblem VMs - Custom VMs Now you start with a job, pay a 30 second penalty, and execute it on an entirely ephemeral cluster. The "get a cluster and fill it with jobs and round up to an hour" model is indeed outdated IMHO. (work at Google Cloud)
Earlier quoted context omitted.
AWS has a free tier for a year
It becomes a problem when you're a student and people expect you to use AWS's free tier for different projects in two different years.
I wonder if Amazon will ever open Lambda up to any Docker image? (I know it's possible to run binaries, but its a bit of a pain to compile with the Amazon AMI, etc.) Being able to have a bunch of `docker run` with any image would be pretty powerful.
Yes. First step was https://aws.amazon.com/blogs/aws/new-amazon-linux-container-... . Layering Lambda's image on top of that to assist people building and testing is definitely on our roadmap.