Live data from Hacker News

Infrastructure From Code

shuttle.rs

31–40 of 44 posts

Re: Infrastructure From Code

#31
post #20

I'm skeptical. Eventually you need to write things for which Rust is the wrong language. How do you get anything to play nice with each other, if you've turned your infrastructure into a magic box? What about preventing things from playing with each other (security)? DNS, WAF, caching, etc.? Are these all just Rust annotations? > To reap the benefits of serverless, you have to build your application as a multitude of…

Right idea, wrong language. At least for me. Not really into Rust a lot and trying to figure it out gives me a headache. I've tried. Also, I think the severe risk here is that anyone who knows Rust well enough to like it for this would probably pull up their nose for devops jobs and find something more interesting to do with their skills.

But, I like the idea of using a statically compiled language rather than some yaml (terraform), json/xml cloudformation, etc.

I'm kind of liking Kotlin and Kotlin script right now (KTS). I might be biased because I write a lot of Kotlin. But it seems to have some unique features that make it suitable for developing your own DSLs.

BTW. I think Gradle's KTS support is actually a bad example of KTS because it inherits a lot of its complexity from when it was still exclusively a Groovy thing (which is of course a scripting language). The Gradle APIs tend to give me a headache. It's never obvious what you need to do. And it keeps on changing as well over time. Which makes Googling the solution really tedious because 95% of what you find is usually a combination of obsolete, wrong, and outdated. Automcomplete via Google is broken. And because Groovy is so dynamic, it also complicates autocomplete in Kotlin Script (which is a major selling point otherwise).

And autocomplete via Google is exactly what makes many solutions in the devops/PAAS/IAAS/configuration management etc. sphere so tedious to deal with. There's just no way around wading through lots of misinformation, product/marketing pages that try to tell you wonderful the shit you are trying to get working is, blog articles, poorly written/structured documentation, etc.

One of our people actually hacked together a little KTS script that generates our github action yaml files. If you've ever dealt with GH actions, they are nice but the yaml sure is fiddly. Having a KTS script that generates those is wonderful:

- A lot less errors that you discover after you commit because if it compiles, it's probably fine. Which means I can tinker with it, commit my changes, and have it work correctly the first time. Amazing.

- No more copy paste 'reuse'. We can have functions and reuse them across different workflows and we can actually reuse bits and pieces that we figured out. I'm always amazed at how much copy paste and adapt is still the leading paradigm in devops automation.

- No more dealing with yaml syntax, it's quirks, and limitations. Kotlin has types, DSL support, multi line strings, string templates, etc. In other words, it's very suitable for this kind of stuff. And when it doesn't have a feature you need, you can just add it because it's a programming language. Not that hard.

This should be a thing in more devops stuff. Terraform would probably be easier with KTS, for example. And I'm pretty sure people are working on that. Though of course the support in Intellij for Terraform is not bad.

Re: Infrastructure From Code

#32
post #19

Earlier quoted context omitted.

One thing I really, really like about the idea is it makes it easier to share full repro test cases if something goes wrong.

Does it, really? That DB still needs tables created and test data loaded. Plus, having to deploy code from the ticket just to be able to reproduce a bug seems like a huge PITA. Give me the local execution!

As long as the reporter can attach table schemas (presumably also created in-code) and data (which can be loaded by the code... presumably) then sure yeah. It's a lot better than having to ask for versions and reproduction cases that don't often come automatically.

> Plus, having to deploy code from the ticket just to be able to reproduce a bug seems like a huge PITA. Give me the local execution!

Ideally the code could be deployed automatically -- perhaps only from trusted/internal parties depending on the product -- so you don't have to actually do that much work. Less than you would if you had to create a database with the specific flags and environment exactly matching those of the reporter.

I dunno, I dream of this sort of reproducible bug report. I don't like the "what version" "what queries" "what errors" back and forth that always seems to happen, even with internal users.

Re: Infrastructure From Code

#33
post #30

Not a single mention here of ansible, Terraform, puppet, chef or any other ways that this is actually done at scale. You can’t readily build infrastructure from Code in a way that allows for solid uptime, redundancy, resiliency, or performance. It sounds like a system like this would be fine for testing and possibly QA but definitely not for production.

Probably because you are conflating Infrastructure as code with Infrastructure from code.

Re: Infrastructure From Code

#36
post #27

Is spinning up a cloud-managed database really infrastructure as code? Seems more like SaaS from code. To me, infrastructure from code would at bare minimum mean spinning up a Kubernetes cluster capable of deploying database and similar containers so that you're not paying a premium to have some blackbox cloud service supposedly managing your services.

Infrastructure has multiple layers. Why does your bare minimum infrastructure start with Kubernetes and not with the OS underneath? Why does it start with the OS and not the firmware configuration? Why does it start with the firmware and not the hardware configuration you order from the vendor?

I mean we have to be realistic in that people will use cloud infrastructure, so it will start with the base infrastructure components they have access to.

Re: Infrastructure From Code

#37

> the code that developers write implicitly defines infrastructure. What your code actually needs in terms of infrastructure should be inferred as you build your application, instead of you having to think upfront about what infrastructure piece is needed and how to wire it up. Your code runs somewhere. On something. In something. At a physical place, with physical cords running around to physical boxes of stuff. The…

I believe the direction they are going is that you have it implicitely define a default infrastructure, and then if you want to customize it you can. This seems reasonable to me. For prototyping or MVP, just go with the defaults. As your product gains maturity you can hire domain experts to fine tune the infrastructure, while keeping it as separate from the business logic as possible

There is no such thing as default infrastructure if you care if your application works. Hiring "experts" later is really just hiring someone to do what the developer should have done, and will probably have to ask the dev to do more work anyway. To an Ops person it's literally a black box. To "fine-tune" the infrastructure, they have to interrogate the developer, and learn only the bare minimum of whatever app stack the developer decided to use. The end result is literally black box reverse engineering, which is wasteful and leads to bad outcomes.

Imagine being a fashion designer making clothes without ever seeing a human being put them on. There's fabric that covers skin, and there's arm holes and leg holes and buttons, so they are "feature-complete" clothes. But just because you pinned up a bunch of fabric on a dummy doesn't mean you know what a person looks like when they put it on and move around in it, or whether a person will want to wear them, or if they'll fall apart after a week. You have to know about different body shapes, how bodies move, evaluate their fit, adapt them to the wear and tear of the real world, and be interested in the outcome. You don't have to be a medical doctor, and you don't have to be an engineer, but you do need to know the absolute minimum of how those different domains interact with clothing.

Re: Infrastructure From Code

#38

Earlier quoted context omitted.

It seems like it'd be all too easy to accidentally create databases and then forget about them when you correct an annotation typo or something. I'm 100% sure that I would do that.

Or, if they try to solve that, you always have the much more exciting possibility of just deleting databases when you accidentally stopped referencing them for one commit:)

There are, of course, a range of other possibilities, such as sending an email when the last reference to a database is removed so that a human can intervene appropriately or having a web app that provides access to the current list of unreferenced databases.

Re: Infrastructure From Code

#39

Earlier quoted context omitted.

I believe the direction they are going is that you have it implicitely define a default infrastructure, and then if you want to customize it you can. This seems reasonable to me. For prototyping or MVP, just go with the defaults. As your product gains maturity you can hire domain experts to fine tune the infrastructure, while keeping it as separate from the business logic as possible

There is no such thing as default infrastructure if you care if your application works. Hiring "experts" later is really just hiring someone to do what the developer should have done, and will probably have to ask the dev to do more work anyway. To an Ops person it's literally a black box. To "fine-tune" the infrastructure, they have to interrogate the developer, and learn only the bare minimum of whatever app stack…

The fact that people use Heroku, Firebase, Vercel, and other infrastructure providers means that there are plenty of developers out there that don't write their code around infrastructure at the beginning.

Re: Infrastructure From Code

#40

> dependencies like databases being provisioned through static analysis in real-time. I'm probably extremely biased (I'm a sysadmin), but this feels like a terrifying way to make your infrastructure entirely implicit and remove anybody's ability to coherently reason about the stuff underneath your application.

> but this feels like a terrifying way to make your infrastructure entirely implicit This exact problem is my main beef with an unfortunately common and popular style of functional programming : stuff so condensed, so abstracted and so focused on describing the problem all the while hiding the actual execution / implementation / solution layers it's quasi-impossible to reason about and understand what actually happen…

You can add tons of structured logs, log runtime configuration... as opposed to trying to "observe" what is going by collecting unrelated pieces of informal logs (the state of the art). People code at high level, tools write instrumented low-level code.
Post reply on HN