Live data from Hacker News

The OpenTF Manifesto

opentf.org

221–230 of 392 posts

Re: The OpenTF Manifesto

#221

I appreciate the letter and trying to work with Hashicorp -- I used to have a ton of respect for Hashicorp. But honestly... at this point... ...just fork it into a foundation. Don't wait for Hashicorp's response. I get wanting to have the appearance of working with Hashicorp, but we've been shown again, and again, and again, and a-fucking-gain that private corporations cannot be trusted to maintain public goods. Only…

Marcin here, co-founder of Spacelift. Even though I strongly believe the OpenTF fork could open up incredible possibilities for the community (I could go on and on about it), it is an equivalent of a civil war. It doesn't serve the community and our only interest is in the continued strength of the community that we continue to build for. Based on my immense respect for what's been built under Hashi's umbrella I'd ra…

> it is an equivalent of a civil war.

Why. It is open source. A fork should be no big deal, and definitely not a “civil war”. I think the community should be quicker to fork open source projects that are not serving the needs of the community.

The corporations are trying to have the benefits of open source without the responsibility. Forking is a normal, acceptable part of open source and we should normalize it.

Re: The OpenTF Manifesto

#222

Earlier quoted context omitted.

MPL is a copyleft license, actually, just like EPL and EUPL. What it is not is a viral copyleft license. Anyone making changes to TF code and productionizing it is expected to contribute back but only the direct changes to TF itself, not the extensions around it. That's my reading of EPL/MPL/EUPL. Does that match your reading?

You're right. MPL is a weak-copyleft license. I mistyped it, I don't know why. Fixed my comment with a note, thanks. However, it still allows your code to be bundled inside a bigger work. The bigger work can be in any license (maybe not GPL, need to check), but MPL stays MPL. This doesn't prevent white-labeling the codebase, though. As far as I read the MPL, the contribution back part is not mandatory, but encouraged…

> As far as I read the MPL, the contribution back part is not mandatory

That's not how I read it.

> the contribution back part is not mandatory

In that case weak copyleft would not be any different from MIT/BSD.

First, assuming you are distributing only the larger work:

"3.3. Distribution of a Larger Work

You may create and distribute a Larger Work under terms of Your choice, provided that You also comply with the requirements of this License for the Covered Software. [...]"

Assuming the covered (OSS) work is distributed as an executable:

"3.2. Distribution of Executable Form

If You distribute Covered Software in Executable Form then: (a) such Covered Software must also be made available in Source Code Form, as described in Section 3.1 [...]"

Finally, 3.1:

"3.1. Distribution of Source Form

All distribution of Covered Software in Source Code Form, including any Modifications that You create or to which You contribute, must be under the terms of this License."

The only wiggle room I see is not triggering the distribution clause by a SaaS-only offering. By the way, EUPL is a non-viral copyleft license that closes the SaaS loophole in MPL/EPL/LGPL.

Re: The OpenTF Manifesto

#224

Earlier quoted context omitted.

You're right. MPL is a weak-copyleft license. I mistyped it, I don't know why. Fixed my comment with a note, thanks. However, it still allows your code to be bundled inside a bigger work. The bigger work can be in any license (maybe not GPL, need to check), but MPL stays MPL. This doesn't prevent white-labeling the codebase, though. As far as I read the MPL, the contribution back part is not mandatory, but encouraged…

> As far as I read the MPL, the contribution back part is not mandatory That's not how I read it. > the contribution back part is not mandatory In that case weak copyleft would not be any different from MIT/BSD. First, assuming you are distributing only the larger work: "3.3. Distribution of a Larger Work You may create and distribute a Larger Work under terms of Your choice, provided that You also comply with the re…

By contributing back, I mean rolling up a patchset (or a tar of your source tree) and sending back to upstream.

Otherwise, I think I clearly noted that the MPL part's source stays available whatever you do by saying "MPL part stays open".

It's not the earliest hour here, so sorry if I can't articulate my thoughts clearly.

You need to keep MPL part's source open and accessible, yet MPL doesn't prevent abuse like permissive licenses, much.

Re: The OpenTF Manifesto

#225

Earlier quoted context omitted.

> Languages and frameworks must remain open or they will wither and die. Terraform is neither a language nor a framework, and I certainly don't think it will wither and die if they transition to BSL. Case in point, docker's revenue grew by 12x once they started taking control of their code and stopped caring about community. Same with postman or nginx or many other companies.

What? Docker is still completely open source apart from the desktop GUI. The engine and (I'm pretty sure) all components are completely free and if anything, they have pushed for the standardization of the container runtime. Buildkit is free, compose is free, no feature is paywalled apart from Mirantis-centric stuff (not part of docker inc) You can absolutely bet that they would get dropped like a rock if they moved…

Didn't Docker actually try it earlier this year, e.g., https://blog.alexellis.io/docker-is-deleting-open-source-ima...?

Re: The OpenTF Manifesto

#226

Actually, can we just kill Terraform? Please? Terraform has a bad design. It's a configuration management tool, first and foremost, and configuration management tools need to do one thing well: fix things. Not just "change state", but functionally, actually fix some software to make it work again. Terraform is really bad at this. It's difficult to configure, difficult to operate, and it likes to find any reason at al…

Would be cool to see your laundry list. I haven’t shared your experience, but I read OPs book on Terraform cover-to-cover before and trying to work with the system.

Oh god I would be writing for hours. Short version, this is not nearly everything:

  - Bad UX
    - Tool does not have interactive mode to provide suggestions or simple solutions to common problems
    - Lack of options or commands for commonly-used tasks, like refactoring resources, modules, sub-modules, etc. (Using 'state mv' and 'state rm', etc is left as an exercise for the user to figure out and takes forever)
    - Complains about "extra variables" found in tfvars files, making it annoying to re-use configuration, even though having "extra variables" poses no risk to operation
    - (NEW) Shows you what has changed in the plan output, followed by what will *actually be changed* if applied, though both look the same, so you get confused and think the first part matters, but actually it's irrelevant.
  
  - Bad internal design
    - HCL has a wealth of functions yet is too restrictive in how you can use them. You will spend an entire day (or two) tying your brain into knots trying to figure out how to construct the logic needed to append to an array in a map element in an array in a for_each for a module (which was impossible a few years ago).
    - Providers are inconsistent and often not written well, such as not providing useful error messages or context.
    - Common lifecycle policy conventions per-resource-type have to be discovered by trial-and-error (rather than being the default or hinted) or you will end up bricking your gear after it's already deployed.
    - The tool depends on both local state and optionally remote state. Local state litters module directories even though nearly everyone who uses it at scale uses modules as libraries/applications, not the location they execute the tool from. Several different wrappers were invented and default to changing this behavior because it has been a problem for years.
  - Default actions and best practices (such as requiring a plan file before apply or destroy, automatically running init before get before validate, etc) are left to the user to figure out rather than done for them (again, wrappers had to solve this).
  - Some actively dangerous things are the default, like overwriting backup state files (if they're created by default).
  - Version management of state is left up to the user (or remote backend provider)
  - Not designed for DRY code or configuration; multiple wrappers had to implement this
  - You can't specify backend configuration via the -var-files option, and backend configuration can't be JSON ... why? They just felt like making it annoying. Some "philosophical" development choice that users hate and makes the tool harder to use.
  - Workspaces are an anti-pattern; you end up not using them at scale.
  - You can't use count or for_each for provider sections, so if you wanted a configurable number of providers (say with different credentials each), tough luck. ("We're Opinionated!")
  - Can't use variables in a backend block. ("We're Opinionated!")
  - Can't have more than one backend per module. ("We're Opinionated!")
  - Lots of persistent bad behavior has been fixed in recent releases, like not pushing state changes as resources are applied, others I can't remember.
  - Global lock on state, because again, ya can't have more than one backend block per module.
  - All secrets are stored as plaintext in the state file, so either you don't manage secrets *at all* with Terraform, or you admit that your Terraform state is highly sensitive and needs to be segregated from everyone/everything and nobody can be given access to it.
  - No automatic detection of, or import of, existing resources. It knows they're there, because it fails to create them (and doesn't get a permission error back from the API), but it refuses to then give you the option of importing them. The *terraformer* project had to be invented just to get a semblance of auto-import, when they could have just added 100 lines of code to Terraform and saved everyone years of work.
  - Not letting people write modules, logic, providers, etc in an arbitrary executable. Other tools do this so you can ramp up on new solutions quickly and make turn-key solutions to common needs, but Terraform doesn't allow this; write it in Go or HCL or get bent.
  - You have to explicitly pass variable inputs to module blocks, so you can't just implicitly detect a variable that has already been passed to the tool. But this isn't the case if you're applying a module; only if you create a sub-module block. This just makes initial development and refactoring take more time without giving the user an added benefit.
  - You have to explicitly define variables, rather than just inherit them as passed to the tool at runtime. Mind you, you don't have to actually include the variable type; you just have to declare *the name* of the variable. So again, it wastes the user's time when trying to develop or refactor, for absolutely no benefit at all.
  - You have to bootstrap the initial remote backend state resources *outside* of Terraform, or, do it with local state, and then migrate the state after adding new resources or using a separate identical module that has a backend configuration. Does that sound complicated? It is, and annoying, and unnecessary.
  - You have to be careful not to make your module too big, because modules that manage too many resources take too long to plan and apply and risk dying before completing. (If you're managing resources in China, make the module even smaller, because timeouts over the great firewall are so common that it's nearly impossible to finish applying in a reasonable time)
  - Tests. In Go.
  - Schema for your tfvars files? Nope; write some really complicated logic in a variable to validate each variable in a different way.
  - Providers don't document the restrictions on things like naming convention for required parameters, so you have to apply to the API and then get back a weird error and go try to dig up some docs that hopefully tell you the naming convention so you can fix it and try again.
  - Terraform *plan* will give you 'known after apply' for values it very easily could tell you *before* the apply, but for whatever reason doesn't. You never really know what it's going to do until you do it and it blows up production.
  - It's very difficult (sometimes near impossible) to just absorb the current state of the infrastructure into TF (as in, "it's working right now, please just keep it the way it is"). Import only works if you've already written the HCL for the resources, and then look up how the provider wants to you to import that resource.
  - Version pinning is handled like 5 different ways, but is still impossible to pin and use different sets of versions when applying different state files for the same HCL module code and values.

That's off the top of my head, there's much more.

Re: The OpenTF Manifesto

#227
post #29
post #28

How exactly do the companies involved plan to fund a fork? It would require at minimum 3-4 full time engineers, and no one is going to do that work for free. It’s also telling that this manifesto blithely suggests TF could become Apache 2, which is wholly untrue.

An earlier version of the manifesto contained pledged resources from each company (you can still find it in commit history). It totalled to ~10 full-time engineers just from founding orgs. It was removed to simplify adding their entries for new pledgees

Do the founding orgs have public disclosure of their finances? It seems the vast majority of them are VC backed companies that probably don't even have two years worth of runway, let alone be in a position to meaningfully commit to funding engineers for five years.

Re: The OpenTF Manifesto

#228

Actually, can we just kill Terraform? Please? Terraform has a bad design. It's a configuration management tool, first and foremost, and configuration management tools need to do one thing well: fix things. Not just "change state", but functionally, actually fix some software to make it work again. Terraform is really bad at this. It's difficult to configure, difficult to operate, and it likes to find any reason at al…

Want to start a GitHub repo? I'll work with you. My list of must-haves for configuration management:

Hierarchical state and provider management. The difficulty of hooking a kubernetes provider to a EKS or GKE provider in a one-shot-apply is pretty terrible. Trying to nest the helm provider under kubernetes isn't quite as painful but still not great and there isn't a way to get the necessary CRD manifests in place for dependencies or resources before they need to be created.

Diffs as a first-class citizen throughout the layers of providers as opposed to situations like helm_release where helm diffs are completely opaque to terraform and especially to tools like Atlantis.

Slightly more of real programming language concepts (pure functions at least), or else insanely good configuration flexibility. Same defaults with simple overrides should be the default for all providers and modules in a standard way. I think deep merge with reasonable conflict resolution is all terraform needs (plus a rewrite of how configuration works in a lot of places), but I want to be able to define a template configuration for e.g. a cluster and be able to instantiate a new cluster with just:

k8s_config = { region = "us-west1" node_config = { machine_type = \ cloud_native_machine_type( \ cpu = "amd", ram = "256G", cores = "16") spot_instance = true } }

And have deep merging successfully override the default configuration with those values, plus that kind of generic function capacity to turn verbose/complex configuration blocks into a simple definition.

Re: The OpenTF Manifesto

#229

Earlier quoted context omitted.

> As far as I read the MPL, the contribution back part is not mandatory That's not how I read it. > the contribution back part is not mandatory In that case weak copyleft would not be any different from MIT/BSD. First, assuming you are distributing only the larger work: "3.3. Distribution of a Larger Work You may create and distribute a Larger Work under terms of Your choice, provided that You also comply with the re…

By contributing back, I mean rolling up a patchset (or a tar of your source tree) and sending back to upstream. Otherwise, I think I clearly noted that the MPL part's source stays available whatever you do by saying "MPL part stays open". It's not the earliest hour here, so sorry if I can't articulate my thoughts clearly. You need to keep MPL part's source open and accessible, yet MPL doesn't prevent abuse like permi…

True, sending back to upstream is not required by the license. But even AGPL does not require this.

Rolling up a tar of your source tree, however, will be required if you get a request from one of your customers by email. The difference from GPL is that you will only have to tarball a portion of the tree that was MPL-licensed. That tar must include your patches, as §3.1 requires.

As I said before, many companies argue that using a private fork of an MPL/EPL/LGPL software in a SaaS does not trigger the distribution clause as customers never get a source or a binary of the program that runs in the cloud. EUPL closes that loophole.

It's also possible that many companies violate EPL/MPL/LGPL terms (knowingly or unknowingly).

One good example of non-viral copyleft working as intended is the Eclipse IDE. There are many closed-source tools that use Eclipse IDE under the hood (the part that Eclipse calls RCP), but there are no commercial forks of the IDE itself because any such company would have to open-source their changes to the fork.

Re: The OpenTF Manifesto

#230

Earlier quoted context omitted.

You may make production use of the Licensed Work, provided such use does not include offering the Licensed Work to third parties on a hosted or embedded basis which is competitive with HashiCorp's products. Read benevolently it's a prohibition from spinning up a service based on HashiCorp's code and undercutting HashiCorp's pricing. On the other hand, if I build a product with HashiCorp-owned BSL'd code, then HashiCo…

My understanding is that the aforementioned companies' licenses are to the same effect, so what is the difference?

Redis is 3-clause BSD, BSD does not have a "your license is void if you sell a product that competes with us" clause. Redis does have enterprise products that are licensed in a manner similar to BSL, but Redis itself is not.

MongoDB and Elastic are SSPL. SSPL approaches the problem like the AGPL; it compels licensees who sell a service derived from the software to make available under the SSPL the source of all supporting tooling and software so that a user could spin up their own version of the service.

There's an argument to be made that SSPL is de facto "you can't compete with us" since it would be more challenging to make a competitive SaaS offering if your whole stack is source available. I don't disagree. However, as distasteful as SSPL is, at least it doesn't grant licensing to a product conditionally on the unknowable future product offerings of HashiCorp.

Post reply on HN