Serverless Map/Reduce • Hacker News Board

Re: Serverless Map/Reduce

#91

Post by vgt » Fri, Nov 04, 2016, 11:17 PM UTC

brianwawok wrote: post #82

ralusek wrote: post #15

This is a screenshot of my google search from 2 days ago: http://i.imgur.com/BNAcSsn.png I've been using Lambda quite a bit, I think it's SO amazingly useful. Tasks that are highly parallelized and CPU intensive can literally be infinitely scaled out. I find it weird that their poster child use case is still always a reactive event like watching S3 and formatting images. There are so many use cases for directly invok…

So google compute setup I did a while back with preemptible instances + a celery queue + some autoscale based on load... The guts to make all that work was 50 or so lines of config. I think my auto scale script was 20 lines or so of Python. I guess the biggest downside was spinning up the new server took about 2 minutes, so for big load spikes it took a bit for it to level out... but with GCE per minute billing, all…

That's essentially what Google Cloud Dataproc gives you (managed Hadoop/Spark):

- Per-minute billing

- 0-to-cluster in under 90 seconds (aim for 30 seconds)

- Pre-emptiblem VMs

- Custom VMs

Now you start with a job, pay a 30 second penalty, and execute it on an entirely ephemeral cluster. The "get a cluster and fill it with jobs and round up to an hour" model is indeed outdated IMHO.

(work at Google Cloud)

Re: Serverless Map/Reduce

#92

Post by yumaikas » Fri, Nov 04, 2016, 11:40 PM UTC

badminton1 wrote: post #52

I do not agree with term serverless. Amazon Lambda is a service, therefore there is a server involved. It's like saying deathless meat, because someone else killed the animal you are consuming.

I think a better term would be FAAS (Functions as a service), since that's what it is. Lambda looks kinda neat as far as tech goes however.

Re: Serverless Map/Reduce

#93

Post by badminton1 » Fri, Nov 04, 2016, 11:48 PM UTC

yumaikas wrote: post #92

badminton1 wrote: post #52

I do not agree with term serverless. Amazon Lambda is a service, therefore there is a server involved. It's like saying deathless meat, because someone else killed the animal you are consuming.

I think a better term would be FAAS (Functions as a service), since that's what it is. Lambda looks kinda neat as far as tech goes however.

Yes, I also think that's a much more accurate term that better represents the concept.

Re: Serverless Map/Reduce

#94

Post by tokenizerrr » Fri, Nov 04, 2016, 11:56 PM UTC

danso wrote: post #50

OT: I teach computational methods and even as much as I dislike teaching/conflating it with web dev, I have included "let's build a web app" because students like building and deploying a thing, and because Heroku has a free tier. I've considered the possibility of having students do things on AWS (beyond web dev), including Lambda, and just expensing the costs. It seems feasible to quickly set up every student with…

AWS has a free tier for a year

Re: Serverless Map/Reduce

#95

Post by radiowave » Fri, Nov 04, 2016, 11:58 PM UTC

elcct wrote: post #32

Is there any AWS Lambda equivalent that could be deployed on bare metal?

No first hand experience with it, but I believe Joyent's Manta system can be, across a cluster of SmartOS machines.

Re: Serverless Map/Reduce

#96

Post by autotune » Sat, Nov 05, 2016, 12:05 AM UTC

danso wrote: post #50

OT: I teach computational methods and even as much as I dislike teaching/conflating it with web dev, I have included "let's build a web app" because students like building and deploying a thing, and because Heroku has a free tier. I've considered the possibility of having students do things on AWS (beyond web dev), including Lambda, and just expensing the costs. It seems feasible to quickly set up every student with…

I would just set up a new account for each student, have them use their own billing info, have them use the free tier, teach them how to set up billing alerts, and let them go to town. They're going to need to learn to take cost into account when working at a real job with AWS so this is the best way to teach them to take accountability.

Re: Serverless Map/Reduce

#97

Post by jamesmcintyre » Sat, Nov 05, 2016, 12:17 AM UTC

aikah wrote: post #80

badminton1 wrote: post #52

I do not agree with term serverless. Amazon Lambda is a service, therefore there is a server involved. It's like saying deathless meat, because someone else killed the animal you are consuming.

Fight the good fight, you're not the only one standing up against this ridiculous moniker. Never back down. Keep calling out the bullshit every time and everywhere this term is promoted. People ask "what's the big deal?" well the big deal is ,calling a service "serverless" is both a lie and misleading for marketing purposes.

The name "serverless" will and is likely doing everything I'm sure it was intended to do... provoke curiosity, signal "this is different than just another PaaS", sound cool and maybe polarize or incite those who get angered over technically less-than-accurate names for frameworks. I'm both surprised and not surprised that HN comments have been so relentlessly focused on this seemingly trivial matter. I can understand that the engineer/scientific mindset would likely take issue with this name but I think the "I'm a human that's grown up in an age of omnipresent marketing-fu." part of you needs to realize that all ideas survive and die on the act of finding a place in your brain- a hook so that next time you discuss with someone their need to scale infinitely/instantly without provisioning containers or an IoT device that only needs logic run in the cloud every 5mins you'll be more likely to remember and suggest: "Well there is this framework with a terribly misleading and inaccurate name that may help you called serverless."

Re: Serverless Map/Reduce

#98

Post by cobookman » Sat, Nov 05, 2016, 12:18 AM UTC

danso wrote: post #50

OT: I teach computational methods and even as much as I dislike teaching/conflating it with web dev, I have included "let's build a web app" because students like building and deploying a thing, and because Heroku has a free tier. I've considered the possibility of having students do things on AWS (beyond web dev), including Lambda, and just expensing the costs. It seems feasible to quickly set up every student with…

https://cloud.google.com/edu/

Google cloud gives free edu grants.

Re: Serverless Map/Reduce

#99

Post by mypalmike » Sat, Nov 05, 2016, 12:30 AM UTC

badminton1 wrote: post #93

yumaikas wrote: post #92

Earlier quoted context omitted.

I think a better term would be FAAS (Functions as a service), since that's what it is. Lambda looks kinda neat as far as tech goes however.

Yes, I also think that's a much more accurate term that better represents the concept.

Great. We now know what color to paint the bike shed. Hoorah, progress!

Re: Serverless Map/Reduce

#100

Post by mypalmike » Sat, Nov 05, 2016, 12:42 AM UTC

badminton1 wrote: post #52

I do not agree with term serverless. Amazon Lambda is a service, therefore there is a server involved. It's like saying deathless meat, because someone else killed the animal you are consuming.

With AWS API gateway + lambda + dynamodb, I maintain exactly zero servers. Zero physical servers. Zero server OSes. Zero sysadmin.

It's perfectly reasonable IMO for that to be called "serverless".