Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Moving my serverless project to Ruby on Rails

frantic.im

Post reply on HN
Print view View original

121–130 of 189 posts

Re: Moving my serverless project to Ruby on Rails

#121

Post by Dunedan »

zerubeus wrote: post #96
Dunedan wrote: post #24

Earlier quoted context omitted.

SAM is able to run AWS Lambda functions locally and to emulate some API Gateway behavior. It doesn't allow you to run all the other services like DynamoDB, SNS, S3, etc. locally. As Serverless applications often rely on such additional AWS services, the ability of SAM to run AWS Lambda functions locally doesn't really help for running such applications locally.

You guys don't even take the time to do a simple search... https://docs.aws.amazon.com/amazondynamodb/latest/developerg... You can have pretty much all the services locally if you want you can even have a step function locally

DynamoDB and AWS Step Functions are notable exceptions to the fact that AWS services are usually not available locally. The vast majority of AWS services isn't available outside of AWS.

Re: Moving my serverless project to Ruby on Rails

#122

Post by marsdepinski »

nitrogen wrote: post #73
awestroke wrote: post #57

Earlier quoted context omitted.

Scaling horizontally is not cheap. A faster language needs fewer replicas.

Rails handled Black Friday and Cyber Monday traffic for an ecommerce company I used to work for just fine. If you are making money, it's worth the cost of 50 lowend VPSes autoscaled (we could have done with a lot less, too). If we were using Java it would have taken us three times the people and four times as long to build the site, and we all would have been laid off.

Not if you hired better people.

Re: Moving my serverless project to Ruby on Rails

#123

Post by grose »

I like using the “serverless monolith” architecture. You use a catch-all API gateway hooked up to a single lambda. You write it the same as any standard HTTP server and let an adapter handle the lambda bits[1]. During development you run it as a regular HTTP server and connect directly to localhost. To deploy it all you need is a shell script that zips and uploads your binary to lambda. All state is stored in DynamoDB. One log stream for everything.

The downside is dealing with AWS events. There’s a few ways to handle it. You can write it into your monolith, toggling behavior with say an environment variable. You can sometimes work around it, for example avoiding the S3 file uploaded event by having clients ping your HTTP API on upload completion. Of course, there are trade offs for all the techniques. You can easily reconfigure a lambda to retry on failure but you’d have to manually implement that client-side. You can also always just add another lambda if you’re ready.

To put it another way: you probably don’t need to write that record to a queue which gets written to DynamoDB, whose stream calls another lambda etc etc. Just do that stuff within a regular old HTTP handler until the fanciness is absolutely necessary.

For personal projects, I appreciate how serverless scales to zero. I don’t have to worry about axing projects with minimal usage.

[1] for example https://github.com/akrylysov/algnhsa or https://github.com/awslabs/aws-lambda-go-api-proxy

Re: Moving my serverless project to Ruby on Rails

#124

Post by marsdepinski »

treis wrote: post #69
awestroke wrote: post #57

Earlier quoted context omitted.

Scaling horizontally is not cheap. A faster language needs fewer replicas.

Compared with developer time servers are cheap. Even for a single developer being 10% quicker at development equates to dozens to a hundred servers.

Yet somehow the popular industry pattern adopted by the industry is to waste ad much developer time as possible.

Re: Moving my serverless project to Ruby on Rails

#125

Post by zerubeus »

Dunedan wrote: post #121
zerubeus wrote: post #96

Earlier quoted context omitted.

You guys don't even take the time to do a simple search... https://docs.aws.amazon.com/amazondynamodb/latest/developerg... You can have pretty much all the services locally if you want you can even have a step function locally

DynamoDB and AWS Step Functions are notable exceptions to the fact that AWS services are usually not available locally. The vast majority of AWS services isn't available outside of AWS.

I have a local tool for every AWS service just tell what are you looking for, for example I use https://github.com/softwaremill/elasticmq to emulate SQS locally

Re: Moving my serverless project to Ruby on Rails

#126

Post by marsdepinski »

joelbluminator wrote: post #112
sodapopcan wrote: post #46

Earlier quoted context omitted.

Boring tech is great if your primary concern is writing business logic. Ideally, this should always be true, but of course the reality is that people's jobs often have them working on products they don't really care for. And of course, it's easy to get distracted by tech anyway. Experience also comes into play when it comes to loving "boring" tech—it took my around a decade before I was sufficiently jaded that almost…

> it took my around a decade before I was sufficiently jaded that almost all I want to do is write actual business logic It's comforting to see I'm not alone. I just wanna ship clean code and contribute value to the business, while all the young bucks in my team are more interested in rewriting our REST apis in Graphql (with all kinds of rationalizations). Looks like the younger you are the more eager you are to expl…

+1. Boring tech just works and provides familiar tools you can build on.

Re: Moving my serverless project to Ruby on Rails

#127

Post by rkangel »

heipei wrote: post #110
rkangel wrote: post #72

For people in a similar situation, I'd strongly suggest giving Phoenix (on Elixir) a look. It's a "boring" Rails style server side rendered web framework, but with a few extra powers: * Elixir is a great functional programming language * When your backend needs to do more complex or longer running work it's just more Elixir rather than complicated architecture involving task queues etc. (because you have the concurre…

I'm curious, and I know this is a random thread to post this question, but how do you handle the fact that the Elixir process might be shut down with regards to necessary persistence of any message queues? Put differently: I consider both message queues and databases in my system to be part of the stateful workloads and the web app part of the stateless workloads (can-be-killed-at-any-time). How do you handle the cas…

The terminology is a bit ambiguous here with the word 'process' so I'm not 100% sure precisely what you're asking. The BEAM (Elixir VM) uses the the term 'process' to describe its green threads/fibres. The BEAM runs as one OS process, with a thread per core of your machine and schedules each of the application green threads onto one of those threads. BEAM processes are cheap and quick to create and come and go regularly. The BEAM itself is very stable and will stay up permanently (potentially even through SW upgrade if you want). Whenever I say process below I mean a BEAM process.

When you use Phoenix, the Web framework creates a process to handle the specific request. By default everything you do happens in that process (give or take some database stuff). If you make a programming error that process will 'crash' (again a BEAM term that is more like an exception) and the user will get an error. No other processes will be affected, regardless of whether they are to do with other requests or anything else.

If you want a work queue, you would create a separate set of BEAM processes (or maybe just one), probably at startup. In your request handling you would send a message to that work queue process asking it to do things. Your request can block (not affecting any other requests) waiting for it if necessary, or it can return so you can get the result later. If the Web request times out or crashes or whatever, that will kill the handler process, but will leave the work queue process alone.

One other note - under the hood it's all message passing between processes, but to the programmer each process is probably just a Gen Server (an abstraction) and the message send and reply is just a function call. But normal Phoenix stuff doesn't even have that because the Web server does the 'process machinery' for handling requests.

Re: Moving my serverless project to Ruby on Rails

#128

Post by dmarinus »

Dunedan wrote: post #28
sheeshkebab wrote: post #19

Earlier quoted context omitted.

If you ever dealt with cloudformation you’d know it’s absolutely enormous PITA. There might be no “silver bullet”, but aws serverless is like rusted bucket.

> If you ever dealt with cloudformation you’d know it’s absolutely enormous PITA. Is it? For me it's working fine. What problems are you facing when writing CloudFormation templates and what better alternative to declare infrastructure do you suggest? Most problems I see are people being not aware of the possible attributes of resources or doing indentation mistakes when it's a YAML template. Most of these problems c…

CloudFormation is slow (custom resources take hours to time out), it's unpredictable (behavior is different for different services). I wish it didn't use stacks (drift happens and can only be fixed manually) and was just idempotent like ansible. It also doesn't come with sane defaults so you have to supply a load of boilerplate code (cdk fixes this using constructs).

Re: Moving my serverless project to Ruby on Rails

#129

Post by dasil003 »

ffdjjjffjj wrote: post #51
thismodernlife wrote: post #32

I have no strong opinion on serverless but I’ve used Rails for 13 years now (in massive $multi-million SaaS products as well as hobby projects) and it still makes me happy. I keep thinking about learning Node and React but I just can’t be bothered because Rails lets me get things done so quickly while also being a joy to write Ruby and Rspec. Regarding hosting, I think this is actually a great time to host Rails apps…

I’m also a big fan of rails, but I’ve experienced a lot of problems scaling it. A lot of the problems ultimately came down to the simple fact that Ruby is really, really slow. At a certain scale you end up forced to develop infrastructure on a different stack to keep up with the CPU load. I never ran into that so quickly when building similar systems with java, go, and C#. I wanna say something nice about rails too s…

This very much depends on the use case. If you are truly CPU-bound than yes, ruby is a bad choice. Similarly, if you are IO-bound or otherwise need a lot of concurrency ruby is not great. I also think it's problematic for large code bases with hundreds of developers. However, there is a huge swath of web apps and APIs that just need to pull a bit of data from a DB or two, munge it together and send it to the client. Rails is perfectly horizontally scalable and plenty fast for this even at scale given Amdahl's law.

That said, there are definitely some self-inflicted wounds people run into here. Of course there are the oft-cited expert beginner problems like n+1 queries and missing indices, but there's a more subtle issue as well: ActiveRecord itself is so convenient that it discourages writing baseline efficient code even when you need it. Basically any time you need to process a medium to large amount of data, ActiveRecord acts like a sort of super boxed type (to use Java terminology) adding a huge amount of overhead. Yes it's true that ruby doesn't even have true primitive types that can achieve the performance of Go or Java, but often times ruby primitives are plenty fast, you just have to be willing to write a bit of extra code, and ActiveRecord as an ORM has plenty of escape hatches at various levels of granularity to facilitate this (eg. pluck, find_by_sql, update_all, execute, etc).

Re: Moving my serverless project to Ruby on Rails

#130

Post by Dunedan »

nitrogen wrote: post #78
Dunedan wrote: post #7

> A lambda publishes a message to SNS, another one picks it up and writes something to DynamoDB, the third one takes that new record and sends an email… Or, you know, just do related functionality in the same AWS Lambda function. While I would probably do the sending of an e-mail asynchronously as well (as AWS Lambda function triggered by a DynamoDB stream), the indirection over SNS to write something to DynamoDB see…

One of AWS's major flaws is that it provides an encouraging UI that almost begs new teams to create a special snowflake manual configuration. The AWS UI should ideally be refactored to be a frontend for an infrastructure as code service like CloudFormation. One of the other problems with AWS+CF though is that it doesn't support every option from every service, e.g. populating secrets was a hassle when I used it.

I like that AWS offers the management console. It can be great for quick prototyping and checking out new services where you don't know yet what to expect. An absolute no-go however is that the management console sometimes uses functionality which isn't available as an official API! One of the most prominent examples for that is probably that the management console does support U2F tokens, while the official API doesn't [1].

CloudFormation lagging behind in terms of features is mainly a matter of priorities and I agree, feature parity from day 1 on would be great. However the way they do it allows them to ship features more quickly, so that's something as well.

In the past CloudFormation support was even worse as the CloudFormation team was responsible for implementing the support for new features of all services in CloudFormation. And with an ever growing list of services and features, that was something which simply didn't scale. From my understanding that changed however, so that most service teams are now responsible for CloudFormation support of their service themselves.

> One of the other problems with AWS+CF though is that it doesn't support every option from every service, e.g. populating secrets was a hassle when I used it.

That's because creating a secret in AWS Secrets Manager is not a control plane operation (which is what CloudFormation usually implements), but a data plane operation (which CloudFormation usually doesn't implement). However in this particular case it has been apparently painful enough, as creating secrets can be necessary to be able to deploy other resources (e.g. for authentication to the Aurora Data API), that AWS implemented this data plane operation in CloudFormation [2]. The same is true for creating parameters in the SSM Parameter Store [3], while it's still not possible to create objects in S3 or items in DynamoDB without custom CloudFormation resources.

[1]: https://github.com/aws/aws-cli/issues/3607

[2]: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGui...

[3]: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGui...

Post reply on HN