Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS AI Stack – Ready-to-Deploy Serverless AI App on AWS and Bedrock

github.com

Post reply on HN
Print view View original

1–10 of 21 posts

Re: AWS AI Stack – Ready-to-Deploy Serverless AI App on AWS and Bedrock

#2

Post by fitzgera1d »

Introducing the AWS AI Stack

A serverless boilerplate for AI apps on trusted AWS infra.

• Full-Stack w/ Chat UI + Streaming

• Multiple LLM Models + Data Privacy

• 100% Serverless

• API + Event Architecture

• Auth, Multi-Env, GitHub Actions & more!

Github: https://github.com/serverless/aws-ai-stack

Demo: https://awsaistack.com

Re: AWS AI Stack – Ready-to-Deploy Serverless AI App on AWS and Bedrock

#4

Post by eahefnawy »

brap wrote: post #3

I don’t get it. How many people need to deploy their own custom AI chat apps over standard models?

This is meant to be a boilerplate or an example of how to build a Serverless AI app on AWS. You can clone the repo and customize it however you like.

Re: AWS AI Stack – Ready-to-Deploy Serverless AI App on AWS and Bedrock

#5

Post by ethagnawl »

I have not read too deeply into this but, do any of these serverless environments offer GPUs? I'm sure there are ... reasons but the lack of GPU support in Lambda and Fargate remains a major paint point for AWS users.

It's been keeping me wrangling EC2 instances for ML teams but I do wonder how much longer that will last.

Re: AWS AI Stack – Ready-to-Deploy Serverless AI App on AWS and Bedrock

#7

Post by isoprophlex »

ethagnawl wrote: post #5

I have not read too deeply into this but, do any of these serverless environments offer GPUs? I'm sure there are ... reasons but the lack of GPU support in Lambda and Fargate remains a major paint point for AWS users. It's been keeping me wrangling EC2 instances for ML teams but I do wonder how much longer that will last.

The big guys are lagging a bit, but there are many smaller parties offering serverless GPU.

I've been a quite satisfied customer of Runpod's serverless GPU offering, running a side project that uses computer vision to detect toxic clouds in webcam feeds of an industrial site.

If you want generative AI, try Replicate, as they have offer a more specialized product.

Re: AWS AI Stack – Ready-to-Deploy Serverless AI App on AWS and Bedrock

#8

Post by ac360 »

ethagnawl wrote: post #5

I have not read too deeply into this but, do any of these serverless environments offer GPUs? I'm sure there are ... reasons but the lack of GPU support in Lambda and Fargate remains a major paint point for AWS users. It's been keeping me wrangling EC2 instances for ML teams but I do wonder how much longer that will last.

I know for sure this has been on AWS's road map for multiple years now. RE:invent is near. Let's see if they can ship..

Re: AWS AI Stack – Ready-to-Deploy Serverless AI App on AWS and Bedrock

#9

Post by scosman »

ethagnawl wrote: post #5

I have not read too deeply into this but, do any of these serverless environments offer GPUs? I'm sure there are ... reasons but the lack of GPU support in Lambda and Fargate remains a major paint point for AWS users. It's been keeping me wrangling EC2 instances for ML teams but I do wonder how much longer that will last.

They use GPUs under the hood for inference/fine-tuning and charge by token. Fireworks will even let you deploy a Lora serverless at the same pricing as base model.

But not aware of any “lambda”-like serverless for any old CUDA workload. Given loading times, it wouldn’t really make sense. Something like CloudRun or KNative for GPUs would be cool.

Post reply on HN