81–90 of 113 posts
Steampipe ( https://steampipe.io ) is an open source CLI to query cloud infrastructure using SQL (e.g. AWS, GitHub, Slack, k8s, etc). It also has a HCL language to define security benchmarks and controls (e.g. AWS CIS, etc). We are a Postgres FDW under the hood with Go-based plugins, so write would be possible, but we've chosen to focus on read only so far. Definitely interested to see how you approach create, update…
We are using steampipe to connect 30 AWS accounts via SQL. In a single call we can query all our services. Using metabase to visualise it all. I highly recommend you give it a go
Earlier quoted context omitted.
We are using steampipe to connect 30 AWS accounts via SQL. In a single call we can query all our services. Using metabase to visualise it all. I highly recommend you give it a go
How are you making the metabase steampipe connection? Are you able to connect metabase directly to a running steampipe/ postgres server instance? or are you doing some ETL into metabase. Thanks in advance (metabase is new to me, but steampipe has been really handy)
Earlier quoted context omitted.
I'm frankly really confused that Terraform is still so widely used for AWS IaC when CDK exists.
One reason is because TF supports many clouds and people like having skills that are not vendor specific
If I did infrastructure in SQL I’d accidentally leave a column out of a join and end up creating millions of dollars worth of machines I didn’t want.
I see a lot of criticisms for not wanting to use SQL to do writes and I think that is misguided. The current state of your infrastructure is absolutely state and SQL is a great language for working with state. While Terraform and all these other "declarative" infrastructure tools are better than what came before them, you're ultimately playing Relation Stitcher by needing to connect the various pieces together. There…
Terraform code is equivalent to the desired content of the database, and running plan generates the required update queries to make it happen.
Seems to me whether you like this or not probably boils down to whether you like SQL or not. For someone who spends most of their time writing type-safe functional code and loving the expressive power of it, the idea of voluntarily injecting SQL into my life seems horribly retrograde. We write whole frameworks to try and avoid having to manually code SQL(!) and every time I have to do it I sigh in frustration at how…
You can not only make queries/views/etc, but you can also write transactions that make the changes, (ie the INSERT/UPDATE/UPSERT/DELETE) that are applied atomically, relying on the SQL "engine" (ie what they're building) to make sure that the relationships (foreign keys, constraints etc) are maintained.
If it's done right, then you can have the equivalent of an ODBC/JDBC driver for it and then use the ORM/framework of your choice to work with the schema.
isnt the big advantage to infrastructure as code the fact that you can version control it? and isnt it notoriously difficult to version control SQL? maybe I am missing something
It helps avoid some scripting / token / API overhead - but I'm wondering what the trade off is in terms of initial setup time.
One general concern I have is around Data (state) consistency of the data after DML operations (insert / update). If the data represents the state of a resource - how do you know if there's a pending operation on it as a result of an earlier Update operation? How is idempotency handled? How are conflicting concurrent state changes handled (one user forces a restart, another initiates a shutdown)? What happens if a change applied to multiple records/resources only successfully applies on some of the resources?
This isn't specific to just this approach though - it's going to apply to any kind of cloud / resource management platform. I'm not sure how they tend to handle it - but a basic single record per resource has limitations around it that mean more work is needed. Maybe you end up with change history records?