Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Automate Your Network

github.com

Post reply on HN
Print view View original

11–20 of 35 posts

Re: Automate Your Network

#11

Post by tguvot »

nu11ptr wrote: post #6

I do network automation for a profession. I build tools (technically compilers) that take a proprietary object model designed for our private cloud and translate that into Ansible (v1) or Terraform (v2) code. At our company, I actually call using these tools in isolation doing it "manually". This is because the largest benefit of automation, I believe, is the abstraction gained from the new object model and being to…

i worked on a product that did something similar for telecoms. had a closed loop automation and graphical designer for object model. it was 10 years ago.

looking today at all the manual work with playbooks/etc, it's astonishing. feels like things didn't move forward at all in past decade

Re: Automate Your Network

#12

Post by iso1631 »

ajsnigrutin wrote: post #7
darkr wrote: post #5

Earlier quoted context omitted.

> 802.1q VLAN tags: access versus trunk I think the parent was saying that these are Cisco specific terms; more generic terms would be "untagged" + "tagged".

Trunk and access ports are like kleenex and bandaids. Yes, technically cisco terminology, but used everywhere.

Absolutely, here's a config from one of my aristas(with bits snipped)

   interface Ethernet1
      switchport trunk native vlan 899
      switchport trunk allowed vlan 801
      switchport mode trunk
   interface Ethernet13
      switchport access vlan 311
And on a Juniper

   set interfaces xe-0/2/1 unit 0 family ethernet-switching interface-mode trunk
   set interfaces xe-0/2/1 unit 0 family ethernet-switching vlan members Mgmt_B
   set interfaces xe-0/2/1 unit 0 family ethernet-switching vlan members Audio_2
   ....
   set interfaces ge-0/0/19 unit 0 family ethernet-switching interface-mode access
   set interfaces ge-0/0/19 unit 0 family ethernet-switching vlan members Audio_2
When Cisco, Arista, Juniper all use access vs trunk it's hardly a vendor specific term

Re: Automate Your Network

#13

Post by jmbwell »

nu11ptr wrote: post #6

I do network automation for a profession. I build tools (technically compilers) that take a proprietary object model designed for our private cloud and translate that into Ansible (v1) or Terraform (v2) code. At our company, I actually call using these tools in isolation doing it "manually". This is because the largest benefit of automation, I believe, is the abstraction gained from the new object model and being to…

There's a push and pull; ansible and terraform both have some facilities for doing what you describe, but of course if you're using both tools, then you wind up where you are, needing yet another layer of abstraction common to both.

In the book, the author presents an approach for storing the object state and organizing the repository for ansible purposes in what is at least as sensible a way as any other I've seen. For installations that might not directly benefit from additional layers of abstraction, managing object model state using ansible's native functionality might well be sufficient.

This is all a legitimate challenge, in any case. Network infrastructure and service instances have some management issues in common, but where they differ, they can differ by quite a bit, in ways that are hard to model at any level of abstraction.

Re: Automate Your Network

#14

Post by tmerse »

nu11ptr wrote: post #6

I do network automation for a profession. I build tools (technically compilers) that take a proprietary object model designed for our private cloud and translate that into Ansible (v1) or Terraform (v2) code. At our company, I actually call using these tools in isolation doing it "manually". This is because the largest benefit of automation, I believe, is the abstraction gained from the new object model and being to…

This sounds interesting, but I am not sure I fully understand. Could an analogy be the object model to loosely correspond to sth like Amazon cdk and the Ansible part being the derived Cloudformation (any other analogy should do, but those are things I understand a bit more although I use quite a bit of ansible, but I am no network Person)? I still don't fully understand the database part. Is it a better way to manage env variables/allows for more flexible input?

Thank you

Re: Automate Your Network

#16

Post by dvno42 »

Hey this is cool! Thanks for sharing your hard work.

I have been living this for the past few years building an automation product[0] and services company to lower the barrier of entry and have tested many of these methodologies. We’ve also written many different runbooks/playbooks for complicated workflows. I’d like to share a couple experiences/opinions:

Netconf and vendor apis are lovely when available and working well. Many devices don’t support this and falling back to SSH (sometimes even telnet) is a must for automation. Imo, you could add value to your book by touching on Ktbyer's Netmiko/Paramiko[1] as well as their nuances (timeouts, dealing with interactive prompts, etc).

AAA is a big component of automation too. Having something in place to handle authn/authz (radius/tacacs) enables consistency for access across vendors. This also enables least privileged accounts and rotation/limited lifetime of creds when used with something like Hashicorp Vault[2]. I think you briefly mentioned secrets management though Ansible vault.

Another technology that may be worth mentioning is Textfsm[3] in conjunction with Netmiko. When we automate workflows for clients, there’s often times where the data we need to parse isn’t easily parsable. Using and expanding on textfsm makes this doable.

Lastly, much automation may only be one firmware change away from breaking. Even with the big vendors, bugs are common that are (ime) low priority to the OEM. Keep this in mind when writing runbooks/playbooks, try to rely on features and output that are unlikely to change across versions.

[0]https://realmhelm.com [1]https://github.com/ktbyers/netmiko [2]https://github.com/hashicorp/vault [3]https://github.com/google/textfsm

Re: Automate Your Network

#17

Post by Cyph0n »

dvno42 wrote: post #16

Hey this is cool! Thanks for sharing your hard work. I have been living this for the past few years building an automation product[0] and services company to lower the barrier of entry and have tested many of these methodologies. We’ve also written many different runbooks/playbooks for complicated workflows. I’d like to share a couple experiences/opinions: Netconf and vendor apis are lovely when available and working…

+1 to textfsm: it is an extremely powerful approach to reliably parse CLI-based outputs. I used to do some IOS-XR device automation when I worked at Cisco - mainly for integration testing - and I (and other teams) used it heavily.

This ties in to your point about how you often need to fallback to SSH or Telnet. For example, a lot of platform-specific data isn’t exposed through standard interfaces, but almost everything is available through a CLI. There are also times when you have no choice but to use the CLI - for example, when re-imaging or reloading a device.

Re: Automate Your Network

#18

Post by nu11ptr »

jmbwell wrote: post #13
nu11ptr wrote: post #6

I do network automation for a profession. I build tools (technically compilers) that take a proprietary object model designed for our private cloud and translate that into Ansible (v1) or Terraform (v2) code. At our company, I actually call using these tools in isolation doing it "manually". This is because the largest benefit of automation, I believe, is the abstraction gained from the new object model and being to…

There's a push and pull; ansible and terraform both have some facilities for doing what you describe, but of course if you're using both tools, then you wind up where you are, needing yet another layer of abstraction common to both. In the book, the author presents an approach for storing the object state and organizing the repository for ansible purposes in what is at least as sensible a way as any other I've seen.…

I'm not using both. The first version of my tool used Ansible. The second version used Terraform. They were written 4 years apart. My users are not devops savvy. They use runbook forms to call into my API giving them a very simple UI that requires almost zero input. The object model includes lifecycling so certain attributes can be changed, etc. and validation done to ensure only a correct network is output. This isn't required by everyone, but it wasn't done out of necessity on how I'm using the tools, but to satisfy the business problem I'm trying to solve (automate network deployment with as few human inputs as possible over the entire lifespan of a client and infrastructure).

I wasn't critiquing the author, but networks inherently have a lot of input data. Much of this is not of concern to the end user, hence why public clouds require almost zero input on the network side.

I agree that my object model is purpose built for our product. It would not work for someone else's network.

Re: Automate Your Network

#19

Post by nu11ptr »

tmerse wrote: post #14
nu11ptr wrote: post #6

I do network automation for a profession. I build tools (technically compilers) that take a proprietary object model designed for our private cloud and translate that into Ansible (v1) or Terraform (v2) code. At our company, I actually call using these tools in isolation doing it "manually". This is because the largest benefit of automation, I believe, is the abstraction gained from the new object model and being to…

This sounds interesting, but I am not sure I fully understand. Could an analogy be the object model to loosely correspond to sth like Amazon cdk and the Ansible part being the derived Cloudformation (any other analogy should do, but those are things I understand a bit more although I use quite a bit of ansible, but I am no network Person)? I still don't fully understand the database part. Is it a better way to manage…

Essentially we have a very specific network topology we are trying to build for each of our clients. The goal is to auto-generate as much of the input as possible, validate that which is given, and allow it to be lifecycled (attributes can change, but only in certain valid ways, objects created/changed/deleted, but only if they aren't referenced by other objects, etc). Due to this, a database is need to store each "object". When the network is "pushed", the database walked and a fresh set of ansible (or terraform for v2) is generated in seconds.

Iow, it is custom set of lego bricks that can only be combined in certain ways to build valid networks. It is propriety to our cloud product which has the benefit of allowing us to abstract things away that others probably couldn't, but the downside of making it entirely non-reusuable for a different use case.

Re: Automate Your Network

#20

Post by totallywrong »

nu11ptr wrote: post #6

I do network automation for a profession. I build tools (technically compilers) that take a proprietary object model designed for our private cloud and translate that into Ansible (v1) or Terraform (v2) code. At our company, I actually call using these tools in isolation doing it "manually". This is because the largest benefit of automation, I believe, is the abstraction gained from the new object model and being to…

Isn't that a lot of words to say that you have a custom set of Terraform modules for your needs? If you're describing a different or better way to do it I'm missing it.
Post reply on HN