Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Welcome to the New AWS AI Blog

aws.amazon.com

Post reply on HN
Print view View original

71–80 of 98 posts

Re: Welcome to the New AWS AI Blog

#71

Post by niklasrde »

deanCommie wrote: post #27
niklasrde wrote: post #11

We have recently set up a TensorFlow assessment function in AWS lambda, and got very close to the maximum allowed size of a lambda function (250MB) with the trained model currently being 85MB, and the TensorFlow libraries and binaries taking up another 140 or so megabytes by default ( I feel like Amazon could do some work in this area to support users to use their own engines and not be bound to AWS AI Platforms and…

It sounds like you're stretching to make the use case fit inside a Lambda. Have you considered extracting the assessment functionality into an SWF activity, invoked from your Lambda via StepFunctions?

No, we haven't looked at SWF at all - might do some reading up on that.

We considered running it on EC2, but the economics just didn't work out for our needs (hundreds of parallel processed jobs with irregular spikes, <5s runtimes per invocation and some others)

Re: Welcome to the New AWS AI Blog

#72

Post by pilooch »

mza wrote: post #51
boulos wrote: post #47

Earlier quoted context omitted.

(Disclosure: I work on Google Cloud). Both providers offer you raw VMs with GPUs and such so you can run popular machine learning frameworks yourself by hand. After that the three providers diverge a bit, and I've not seen a good writeup myself. Roughly: - Google has both a hosted TensorFlow (Cloud ML) as well as specific, pre-trained models you can simply use (Cloud Vision, Cloud Speech, etc.). For an easy to use in…

GM for AI at AWS here: actually - we like TensorFlow quite a bit, too. We provide a machine image with TF, MXNet and others pre-installed, along with Keras, CPU and NVIDIA divers, and other libraries for deep learning. We just added Ubuntu support too: https://aws.amazon.com/blogs/ai/the-aws-deep-learning-ami-no...

https://aws.amazon.com/marketplace/search/results?searchTerm... Some more AMIs for deep learning.

Re: Welcome to the New AWS AI Blog

#73

Post by dirtyaura »

nielmalhotra wrote: post #44

Quick slightly unrelated question: Does anyone have a comparison of using Google cloud services vs AWS for machine learning? I'm planning to pick one, and I was leaning towards Google Cloud Services because of the TensorFlow support and the fact that Google is big on ML, making it likely that it's something that Google will support and be good at. With this blog post, I'm not sure.

If you are just learning Deep Learning, using Keras on top Theano on a single GPU is a good option. AWS has p2 instance that is used in Part 1 of Jeremy Howard's excellent fast.ai MOOC. Tensorflow becomes more useful when you have multiple GPUs.

Re: Welcome to the New AWS AI Blog

#74

Post by boulos »

tekoyaki wrote: post #46
nielmalhotra wrote: post #44

Quick slightly unrelated question: Does anyone have a comparison of using Google cloud services vs AWS for machine learning? I'm planning to pick one, and I was leaning towards Google Cloud Services because of the TensorFlow support and the fact that Google is big on ML, making it likely that it's something that Google will support and be good at. With this blog post, I'm not sure.

Curious too, esp on the new Cloud ML: https://cloud.google.com/ml/ I guess it's more comparable with Google Cloud Functions: https://cloud.google.com/functions/

Hmm. Cloud ML is "serverless" in a sense, but it's backed by VMs running CPUs or GPUs (up to you). Cloud ML is "hosted TensorFlow" and does a lot for you. The set of pre-trained model services (Cloud Vision, etc.) are quite a bit different.

For more info, see our new blogpost (discuss here: https://news.ycombinator.com/item?id=13697666).

Again, important disclosure: I work on Google Cloud.

Re: Welcome to the New AWS AI Blog

#75

Post by doppenhe »

niklasrde wrote: post #11

We have recently set up a TensorFlow assessment function in AWS lambda, and got very close to the maximum allowed size of a lambda function (250MB) with the trained model currently being 85MB, and the TensorFlow libraries and binaries taking up another 140 or so megabytes by default ( I feel like Amazon could do some work in this area to support users to use their own engines and not be bound to AWS AI Platforms and…

Have you tried using Algorithmia for this? We get around a lot of these limitations that Lambda has (you can also call us from lambda and we can host most of your DL models https://algorithmia.com/developers/algorithm-development/mod...).

Disclaimer: work @ algorithmia

Re: Welcome to the New AWS AI Blog

#76

Post by mmcclellan »

nielmalhotra wrote: post #44

Quick slightly unrelated question: Does anyone have a comparison of using Google cloud services vs AWS for machine learning? I'm planning to pick one, and I was leaning towards Google Cloud Services because of the TensorFlow support and the fact that Google is big on ML, making it likely that it's something that Google will support and be good at. With this blog post, I'm not sure.

On the GPU side of things, I can confirm that AWS p2.xlarge has worked well for me. It has one Tesla K80. Azure's offering is similarly priced. Back in November Google Cloud announced P100 GPUs would be available soon; that will be interesting.

Re: Welcome to the New AWS AI Blog

#77

Post by saosebastiao »

I know ML is the big cheese right now, but doesn't it seem like a bad use case for the cloud? Consider:

1) Training ML models does not require network access, which is one of the biggest competitive advantages of the cloud.

2) Training ML models is typically a batch process, which benefits minimally from the scale-on-demand model of the cloud.

Since the cloud premium is a significant exchange for the value that it adds, I don't see this being a big win for cloud providers. I can't help but think that if I were making use of extensive machine learning with continuous training, I'd have it training models on a local bare metal cluster statically scaled to my application's demand with minimal network connectivity needs. And then ship the serialized trained models to the cloud. The potential cost difference is huge.

Re: Welcome to the New AWS AI Blog

#78

Post by tcas »

niklasrde wrote: post #11

We have recently set up a TensorFlow assessment function in AWS lambda, and got very close to the maximum allowed size of a lambda function (250MB) with the trained model currently being 85MB, and the TensorFlow libraries and binaries taking up another 140 or so megabytes by default ( I feel like Amazon could do some work in this area to support users to use their own engines and not be bound to AWS AI Platforms and…

I've hacked around lambda quite a bit (I think the compressed size of one function is a tad under the max allowed). My hacks I remember are:

- Run strip all .so libraries -- many aren't stripped fully

- In Python I manually deleted sub packages of numpy/scipy I didn't need

- If you're loading large models at initialize, numpy load routines are _much_ faster than cPickle. Have it load at module initialization, not during each invocation.

I should really write a blog post about my experience with it.

At a certain point I decided I was doing something that lambda really wasn't designed for -- I'm looking at migrating off, but the current implementation makes capacity planning super easy. Provisioning 1000 machines with 1GB of RAM for 15 minutes every day to read off a queue isn't a trivial problem.

(Also, if anyone from AWS is reading, being able to limit the max concurrency of a single function vs account level limits would be super useful).

Re: Welcome to the New AWS AI Blog

#79

Post by btown »

saosebastiao wrote: post #77

I know ML is the big cheese right now, but doesn't it seem like a bad use case for the cloud? Consider: 1) Training ML models does not require network access, which is one of the biggest competitive advantages of the cloud. 2) Training ML models is typically a batch process, which benefits minimally from the scale-on-demand model of the cloud. Since the cloud premium is a significant exchange for the value that it ad…

> does not require network access

> benefits minimally from scale-on-demand

Neither of these is true when dealing with terabytes of data (or more, if you're working with image/video corpora). Many AI/ML problems have stages that are trivially parallelizable - if you can divide your problem into iterations where a subgraph of nodes communicates internally, then sends/receives updates to other subgraphs, it's very similar to an iterative map-reduce algorithm, perfect for networked cloud systems.

And as you're tuning your hyperparameters, you don't know what the performance characteristics are, and you will absolutely want to run experiments in parallel, until you find the right settings that you'll use in production. You'd need to invest in a LOT of redundant bare metal to have that capability. As Netflix puts it in this presentation, the key to effective machine learning is iterating often, and that means having a lot of parallelism to bring to bear. https://www.infoq.com/presentations/machine-learning-netflix...

Re: Welcome to the New AWS AI Blog

#80

Post by btown »

tcas wrote: post #78
niklasrde wrote: post #11

We have recently set up a TensorFlow assessment function in AWS lambda, and got very close to the maximum allowed size of a lambda function (250MB) with the trained model currently being 85MB, and the TensorFlow libraries and binaries taking up another 140 or so megabytes by default ( I feel like Amazon could do some work in this area to support users to use their own engines and not be bound to AWS AI Platforms and…

I've hacked around lambda quite a bit (I think the compressed size of one function is a tad under the max allowed). My hacks I remember are: - Run strip all .so libraries -- many aren't stripped fully - In Python I manually deleted sub packages of numpy/scipy I didn't need - If you're loading large models at initialize, numpy load routines are _much_ faster than cPickle. Have it load at module initialization, not dur…

Would love to see that blog post!
Post reply on HN