Ask HN: How to deploy an AI App?

#1

Post by gyeonggi » Wed, Nov 21, 2018, 5:11 PM UTC

My mate and I are thinking of deploying a website (maybe running Node & Express), where users can upload an image. That image will be passed to a python script in the same environment, which will contain the AI logic / neural network. Then we want the image to be sent back to the front end.

We are looking for the lowest-cost solution as this is just a hobby project, maybe even a solution where the server starts up on demand when someone tries to reach the website.

How would you deploy an application of this sort, using what cloud service?

Re: Ask HN: How to deploy an AI App?

#2

Post by SkyTreasure » Wed, Nov 21, 2018, 5:22 PM UTC

If you haven't brought aws yet. They provide 1 year free service, check if its available.

Then deploy your server side API code using lamda and serve users from that layer(Read abt serverless architecture, u don't need an ec2 instance). Which is relatively cheap, its pricing is based on number of times it runs, if i remember correctly they give 1 million free call per month i guess, even if its not, its pricing is relatively cheap.

And store the images in s3.

Re: Ask HN: How to deploy an AI App?

#3

Post by pplonski86 » Wed, Nov 21, 2018, 8:18 PM UTC

How big is your model (neural network)? How many RAM you need to load the model? How big are images?

Do you need a website running (same pages with html)? Or just need API to the model?

Re: Ask HN: How to deploy an AI App?

#4

Post by gyeonggi » Wed, Nov 21, 2018, 9:17 PM UTC

pplonski86 wrote: post #3

How big is your model (neural network)? How many RAM you need to load the model? How big are images? Do you need a website running (same pages with html)? Or just need API to the model?

Size of model is unclear at the moment, but will likely need 4 gigs or RAM or so?

We will only be doing inference with this particular app, the training on another machine.

Images will vary based on what users upload, but larger ones a couple GBs.

Would be ideal to have a website running, to be user friendly and present all of this.

Re: Ask HN: How to deploy an AI App?

#5

Post by gyeonggi » Wed, Nov 21, 2018, 9:18 PM UTC

SkyTreasure wrote: post #2

If you haven't brought aws yet. They provide 1 year free service, check if its available. Then deploy your server side API code using lamda and serve users from that layer(Read abt serverless architecture, u don't need an ec2 instance). Which is relatively cheap, its pricing is based on number of times it runs, if i remember correctly they give 1 million free call per month i guess, even if its not, its pricing is re…

Something along the lines of this article? https://aws.amazon.com/fr/blogs/machine-learning/how-to-depl...

Re: Ask HN: How to deploy an AI App?

#6

Post by pplonski86 » Wed, Nov 21, 2018, 9:30 PM UTC

gyeonggi wrote: post #4

pplonski86 wrote: post #3

How big is your model (neural network)? How many RAM you need to load the model? How big are images? Do you need a website running (same pages with html)? Or just need API to the model?

Size of model is unclear at the moment, but will likely need 4 gigs or RAM or so? We will only be doing inference with this particular app, the training on another machine. Images will vary based on what users upload, but larger ones a couple GBs. Would be ideal to have a website running, to be user friendly and present all of this.

Looks like AWS lambda can be too small for the job.

Have you checked https://www.tensorflow.org/serving/ ?