Viewing profile — TheHydroImpulse

CDK outputs CloudFormation templates that you can pass to any other tool if you wish. CDK is technically just a glorified shim over CFN (although 1000x better but you do get caught…

still very different but amazon has its influences in many parts of the company now.

If anyone notices a problem you'll likely need to write a COE, there's no way to get around that. Not updating the status page absolutely doesn't get you out of that task. COE also…

yes, that's a really good way to grow your comp. You have actual leverage when joining a new company that you can use to negotiate. Obviously you can't really do it too often but w…

Companies usually like to downgrade when hiring. Amazon does it a lot. So even if you qualified as an L7 they may hire you as an L6 (sometimes with L7 pay) and you need to go throu…

Github has been a well known Rails monolith. Absolutely they have services that power all sorts of stuff (they have a few blog posts on it) but it's just impractical to even start …

This just isn't possible. No company in their right mind would take their monolith and rewrite a bunch of stuff just to please a couple randoms on the internet. I think you're dras…

when you have to manage a large number of services with a bunch of different devs touching things, Docker is almost required to get a consistent development state. There's literall…

As someone without a degree, getting an interview at the Big 5 was quite easy. Passing those interviews is another thing entirely but having the right experiences in your resume ha…

L3 at other companies is an L4 at Amazon (I believe only contractors or part-time employees can be < 4). L5 also has a much wider surface area and the gap from L5 to L6 is quite bi…

The biggest issue with PubSub and Amazon's alternative is the cost. Being capped at a per-message cost would be a no go. If you can get away with using PubSub or the like it would …

We started deploying our Kafka cluster as a set of N EC2 instances but we started running into a bunch of issues (rolling the cluster, rolling an instance without moving partitions…

NSQ has been (IMO) far far easier to operate than Kafka has been. With Kafka you need a Zookeeper cluster in addition to your Kafka brokers. Not to mention developing against NSQ i…

A bit of an older article but still useful: https://segment.com/blog/scaling-nsq/

Engineer @ Segment NSQ has served us pretty well but long term persistence has been a massive concern to us. If any of our NSQ nodes go down it's a big problem. Kafka has been far …

A bunch of people at Segment have looked into it. We manage our infrastructure with Terraform and operationally IMO Kubernetes introduces too much complexity compared with ECS and …

One thing we experienced at Segment was the fact that we needed to quickly handle a surge in volume but couldn't overload partners. Essentially we wanted something that scaled up q…

CPU + memory autoscaling has worked pretty well for most services at Segment. We did have to build a custom queue-based autoscaler that feeds into ECS for our integration stack but…

At a high enough scale, you have a lock-in _somewhere_. Spending time trying to abstract yourself from any lock-in can be wasteful.

I'm talking about _my_ use case for using AWS. I'm sure other people have similar requirements. We manage hundreds of servers, process over 50 billion events/month and losing data …

a) I don't really have a good enough sample size but I'd imagine a lot don't. The biggest selling point of AWS is everything around it. You don't just get EC2, you get Route53, ELB…

Unfortunately, mixing and matching ends up really complicating things especially with security in mind. Many people run within a VPC and bridging to another private network is, wel…

Oh, for sure. A lot of startups don't need that ability. I work for a pretty infra heavy startup so AWS is simply required at this point. But we've hit AWS capacity limits during t…

Until you need to add another server, or 10 or 100... Not to mention it's another set of skills you need to have. It's a tradeoff. (I'm not talking about "we may need 100 servers n…

Latency is around 80ms-200ms per invocation from what I've seen. Definitely not suitable for any real-time game.