Read blog
De-risk your SD-WAN rollout with network digital twin technology.
read more

Next-Generation Network Operations

IP Fabric's Joe Kershaw and Daren Fulwell host a panel of stellar network automation experts to discuss a number of themes in Next Generation Network Operations

Transcript

Thank you everyone for joining us, and thank you to our, esteemed panel for for taking the afternoon or the the morning off to, to spend this time having a bit of a discussion with us. Us. This is the the second part to the community fabric ask community fabric anything, roundtable series, and we've got some, movers and shakers from the network automation community with us. The the themes are fairly open. We're gonna leave it as a as a wider discussion.

I will personally be watching the chat, looking to in introduce some of the into the conversation if, if Jeremy and John let me get a word in Edgeways. But general themes here, we're gonna pass around for some quick introductions, a little bit about people's passion projects, what they're focused on at the moment. But some of the themes we want to tease out of this is why are we doing it? What what is network automation at such an advanced level? What's the the evolution and the path for network automation for next generation network management, and and who really cares?

Like, what do users want to actually consume? What's the next point, and what's the reason why? So we've got 58 minutes for a flowing discussion. Everyone on, on the audience, please feel free to fire in questions as they come to your mind. But I will, let's pick a random.

John, do you wanna start us off with a bit of an introduction and an idea of your passion projects at the moment? Yeah. Great. Hi. So my name is John Capobianco.

I'm a senior IT integrator and planner for the Canadian House of Commons. That's the parliament of Canada. And, I started on my automation journey probably 5 years ago, maybe 4 years ago. And, you may know me. I've written a book about that was a very big passion project of mine for a long time.

But more recently, I've spun up a an open source network automation Python based project called Merlin that's that really keeps me busy and, I'm really excited about. So I'm really happy to be here. I think this is a great panel, and, thanks for sharing your time with me. Joe. Jeremy, do you wanna follow on?

Yeah. Yeah. Thanks. Hi. My name is Jeremy Schulman.

I'm a principal software engineer at Major League Baseball, and, I've been focused on, what I would consider modern network automation since about, 2011, 2012, so I'm almost a decade in. I think I'm the 1st person who ever applied to Ansible to networking when, Ansible was 4 people as a start up here in Raleigh, North Carolina where I live. And my passion projects are are my work projects. You know, I'm mostly focused on chat ops as well as network observability. So, you know, happy to be part of this panel and look forward to, you know, talking about those things and answering folks' questions.

Great. Thanks, Jeremy. Alex. Yes. So I'm Alex Gittings.

I'm a professional services consultant at a UK firm called Axions. My passion project is mainly chat ops and management tools, per se and integrating management tools into into networks. Great. Thanks, Alex. Do you want to, finish off, Darren?

Sure. I mean, Darren Forwell. I'm, IB Fabric's product evangelist. Hey. My job is working with folks like this.

So, I I've got the best job really, because I don't have any specific projects. I just spend a lot of time talking to customers, talking to partners about how to integrate, different platforms together and make them do cool stuff. So, what better job than that? Hey. But, nice to be here.

Cool. So to try and tease it off, there's no question coming in. There's there's already reviews being pressed, John, for Merlin. So you're gonna be a busy man after this session. But to just start it off, I guess, somewhere interesting to to kind of, kick us in would be like, what is network automation to you?

And the reason I ask is we see different people talking of different themes, different technologies, different approaches, and it's almost this kind of self perpetuating bubble. Hopefully, not a bubble that will pop, but what what is network automation? Why are you really passionately chasing down this path? I'll leave it open for anyone to respond. I can see Jeremy is itching to to Yeah.

Yeah. I I Go go go go. Fine. We we know you have, have views on this one. Yeah.

So the way I look at it from a really high level, you know, anything that I look at doing regardless of whether it's a commercial product or or a passion project is is solving 1 of 2 problems. Either you're going from, you know, an to ching kind of, scenario. Meaning, you know, you need to get some project done and you need to get it done quickly. You know, you know, the the business owners or the stakeholders are like, gotta get something done, wanna get it done, you know, or it's going from, you know, oh, crap, you know, something's wrong to all clear. We've we've figured out the root cause and we can fix it, you know.

So for me, I apply that that kind of thought process to anything anything that we do, whether it's a commercial product. So, you know, some people look at network automation as commercial products. You know, it's anything that isn't CLI typing is is network automation to them. And and some people, consider, you know, writing, you know, tools like Merlin, you know, automation programming or or automation, engineering. So it's a broad spectrum.

You know, where I work, there's a broad spectrum of people and talent. So I I get to see that full spectrum. So, I mean, the the way I've started looking at this more and more is that, obviously, from from a network engineering standpoint, we get pushed on this network automation being about deploying configuration and and, you know, consistency config and all of those sorts of good things. But but it's it's so much more than that though, isn't it? Because really what what I know I know you do a lot of this, Jeremy, and and from what you guys have been saying as well, it's as much about how you're reaching out into the wider ecosystem and bringing things together to make network operations better.

Right? As as and there's so many different so many different ways you can take that, I suppose, but it all has to start somewhere. I know, John, Merlin, you know, it's it kinda stems from that really, and and tell us more. Yeah. So I I agree with Jeremy, and I I think that network automation is is I think we have to frame it in a modern context.

I mean, there's bat files and scripting and the old, IPSLAs. I mean, it's been around for a while in different formats, but I think what we're talking about is abstracting, right, important information from configuration management. And I think we're part of a larger stream of, this shift from, I don't know, let's say, 90, 10 to 80, 20 human to machine traffic to the the almost the complete inverse where it's 80 or 90% machine to machine or m to m, and that's what I really wanna do. You know, my little secret is I'm pretty lazy. I mean, I'm a lazy guy, and if I can take a few minutes extra to write some code and let a machine talk to the machines.

Right? A a Linux box is gonna talk to 200 routers. Well, we have a common language in in JSON or even XML that like, these guys have talked about chatbots, And I think it's incredible that when we move to this idea of, can I get machines to talk to machines? Well, there is a common language. I can get JSON structured data from a router, then I can send it upstream as JSON to whatever you want, SharePoint, Discord, Webex, Zoom.

I've gone as far as to sending it to Twilio and making phone calls as we did together, Darren. Right? So Yeah. Yeah. I think it's about using your imagination, and and the tools have finally arrived for us to work like developers and develop solutions.

Right? Jeremy hit hit, like, everything he looks at, everything I look at, it's an automate first principle. Right? I don't want a GUI. I don't want a CLI.

I want a restful API, or I want some Pythonic libraries. I want better ways to manipulate these these constructs we work with. Yeah. I find I find that the the more I'm going into it, I'm merging tools together to achieve what I need to achieve rather than building a full system. It's it's the system is a bunch of different tools that do one thing and merging them together to get the whole solution, in in different in in different scenarios.

An interesting question in from from Abdul Hameed here. And I guess this touches for for the the things you just mentioned there, John and Alex, are are elements of different tools that maybe have been scripted or built up in a community or anything. Abdul Kameed is saying within his last 3 years of network automation, he's seeing all of these companies springing up. So more commercial products, more vendor type things and companies. And he's asking, where do you see this going in the future?

And I guess a good way to frame is, do you start to expect something different from a commercial vendor to what you expected in the past? Yeah. I mean, you know, I I spent 20 years in the vendor land before, you know, joining MLB, which is my first, you know, commercial or, you know, customer side of the house. And so, historically, network vendors, I think, have have, not done very well. I think we'd all agree When it comes to network management solutions of all kinds, I think there's there's no disagreement there.

So, I think that the opportunity for start up companies, you know, to tackle different aspects of network management. I I don't like using the word network automation because network automation is a way to a means to an end. Yeah. I mean, people want, you know, network management solutions or life cycle management solutions or service management solutions. So, I'm hopeful that there are gonna be more companies that are building quality software products first rather than trying to take something that's either been a legacy product and slap a a a fresh code of a REST API on top of it and call it, you know, done, which I I've I've seen that, quite a bit, but things are getting better, I I I would like to believe.

For for me, it's it is that interaction, though, isn't it? It's the the fact that that you're gonna have multiple systems doing different parts of the of the of the network operations space and be able to to move data around to share it rather than replicate it in lots of different places, be a be a switch to to take information from one place and push it into another so that you're able to make make everything consistent. You know, having rest APIs starts to give us the capability to do that in a structured way. John, you've already mentioned the idea of having JSON data coming from one one system and being able to play out into another in order to to move that data around and get the visibility. Ultimately, it's about bringing all of this tooling together to to build a service.

Right? Not not just not just moving packets from one place to another, but being able to understand why you're moving packets from one place to another and what benefit that has, not even just to the folk who operate that network, but the people who use it for to to deliver applications. So the I I think the scope is limitless for this. Right? Yeah.

But, John, I mean, I would really like to hear you talk about the stakeholders and consumers of the data that you're, producing with Merlin because, I mean, you and I have had, some kind of conversations about what you like to call business ready documents, and that really kind of stuck in my mind. So, you know, a lot of times we get focused on the tools and the technologies, but I'm I'm really curious, like, okay. So you've built this tool and you have this technology, but who's benefiting from it and and the impact that it's creating is is really what I'm interested in. Well, I'm I'm happy to talk about that, and I think it it sort of stems from something that stuck with me from my private sector days with an insurance company. We we spent a lot of money on some high end equipment, and, someone in the finance department said, okay.

Can I get an inventory and a spreadsheet from that big, you know, router you just bought? And I said, well, I I mean, I can get you a show inventory command. Right? And and there was a bit of a disconnect there. Like, they they don't want a CLI output of a show in.

They want a spreadsheet. Right? The 19 eighties killer app. Right? The spreadsheet that just changed the world.

So to your point, I have to track every dollar we spend. It's all public money, and and we have we don't sole source. I could buy it from any number of different partners and platforms or vendor direct or whoever wins the the the contract, but we have to be accountable. So there was I don't wanna say a panic, but there was a real appetite for asset management information. And it it makes you seem simplistic.

Right? I'm not doing BGP neighbors and OSPF. Right? I'm just getting an inventory, but normalizing it across every platform. Right?

So an f five load balancer has SFPs and blades, Cisco, every flavor. Right? The 4 different operating systems we have. We're using REST and using parsers and different technologies. I'm able to normalize all of these random datasets into assortable, searchable, filterable spreadsheet, my spreadsheet had 8,000 parts in it that I was able to present in one centralized spreadsheet that we also sent to SharePoint.

So now I don't need to send out get links. I don't need to send anything, but here's the SharePoint document library hyperlink to finance and to compliance and to audit and to to the business side of my house that that don't want CLI output and, you know, they don't even know how to read that stuff. They want part number is in this building, in this device, and and here's the serial number. So then we went even further talking about moving data and connecting to APIs. Well, certain companies have APIs where I can send a serial number to it, which is easily done.

I have the serial number as a JSON key. Well, now I get my contract information back per part. So now I have 2 reports, my massive inventory list, and then a contractual state per part that the business can consume, and we can make business decisions on. Oh, look at these parts are end of life in 2 years, or this part is already end of life. Oh, this part this product has a a p cert alarm because I'm connecting to the security APIs that say, oh, based on that part number, I know you have an open vulnerability.

Like, it's about connecting, like you said, business ready documents. There yes. There's technical documents. I have the JavaScript, but management, operations, finance, security, compliance, just a general report that my CTO can read. Right?

He's very good at reading a CSV file, and we have then then you then you turn it over to your Excel wizards. Right? Pivot tables and bar graphs and trending and charts, and we've fed it into Power BI now, where now I can trend on when things are coming end of life or what parts are moving around my network. Right? So I I'm really I'm I'm I can talk about this forever.

Right? But So I was just gonna say, I mean, that you you I guess the point is that that's generated automatically. Right? So so Automatically. You've got that that constant refresh of of that data available.

You've not got that situation where every year, the big audit week comes up and everyone has to go off and collect all the data and piece it all together and bring it all back because it's there already documented automatically. But you hit the nail on the head there and, like, take those as your inspiration for infrastructure as code. What do I do once a year, some big bang thing where I have to go track down every part number? What do I do 3 times a day? What do I do 10 times a week?

Pick those things and pull them off the vine and automate them. Right? Have you found any issues with you said you you used the serial number to find contractual information. Have you found any issues whereby a specific vendor does not support that specific feature, so you've not been able to implement that because that's that's quite a touchy top subject is that a lot of vendors are talking about automation, but it's not necessarily the device automation that we're trying to achieve. And they're not the you know, the APIs or the programmability around a service or a contract is not there throughout the vendor, not just on the configuration management of a device?

Yes. It's we've gone as far as our Cisco platform. Right? They have the serial to info API, and they have the product security and API, and they even have a recommended release API. So that trilogy of APIs from that particular vendor, are are really enhanced our our like, a 360 view.

Like you said, it's not just about and this is I'd make this point in my book. DevNet makes the point. I have heard Darren make the point. Don't get stuck on configuration management. Right?

I wanna push 10 VLANs. Like, I wanna push a route. That's that is another level of complexity if you're just starting in. Right? Like, I I don't know where everyone started, but I didn't have the luxury of, like, I was thrown into the fire.

We had a massive change, and, we decided to automate it. And it was a mistake. Right? Like, looking back, it was a complete mistake for us to say, let's do this production complex change across 30 routers automatically as our first attempt. Now I yeah.

I've learned a lot since then, but I don't know if you could Alex, like, what was the first thing you think you really automated it, and would you do it again that way, or what have you learned? So the first thing I automated and what got me into automation was had, like, a mentor when I was working at Cisco, and we were working on, UCP technology being able to deploy NFE's network function virtualizations to a cloud based edge an edge compute platform. So instead of putting a router, you put you put a a a server, and then you have your virtual network work functions. And it was getting a CSR 1 1000 v booted up in OpenStack automatically, config. That was my, ah, it that's cool.

And then I've been doing it ever since at Ansible, Python, and so forth. So that was kinda my first, but it wasn't like a production change because I was I was that was like a mentored project. But it was pretty cool, but not the same kind of fire fire pit that that you you had. Yeah. My, my my first experience, my first moment experience really kinda set the course on my my point of view, which was we, we ported, Puppet to run on Juno switches.

Right? And this was not done for the network engineers. Like, this the the the request was not as a network engineer who uses Puppet. You know, I wanna manage VLANs and lags, you know, and interface changes on switches. It was for the application developers who use Puppet to deploy their apps.

Mhmm. And they wanted to articulate the changes to the network that were required without involving the networking team. And so that was the big winner. That was the big that, you know, the the stakeholders for valuable business ready network automation is not for the network engineers. I mean, you know, we're talking to an audience of network engineers who want better tools to help them do their jobs, and and that's great.

I mean, there's nothing wrong with that. But when I was involved with, you know, business decisions, like, do will this business spend $20,000,000 on buying this, you know, network gear? It was pinned around not the network engineer's experience with network with network automation. It was at a much higher level at the business, which is is, you know again, I look at who are the stakeholders and what are the impacts to the the network automation. And and that was my starting point.

You know, that was back in, I guess, 2012 when when we did Puppet for Junos. And then everything else kind of fell forward from that. Oh, so good. And I think this this is an interesting point you made there, Jeremy, is that the actual major purchases around tech, around softwares, around whatever is is usually sitting at a business level. It's not necessarily based on the engineer's experience of it.

And then taking what you mentioned, John, about it doesn't matter what the kit is. I want a standardized business readable interface, whether it's an Excel spreadsheet or it's something similar. So the the thing that I'm seeing a lot more with our customers, with our partners, is that network automation is a capability, regardless of how it's delivered or what the approach or whose skills or whatever else. It's a case of abstracting the complexity to allow network engineers to deliver better interfaces to the business, but also allow the business to make their decisions more freely on which kit they want to go with, whether it's price pressure, time pressure, delivery times, which are obviously crucial at the moment, whether that would influence a vendor selection. But it allows businesses to make a decision at a different pace.

And based on more important aspects to them other than we've got a team who know how to operate this box. Yeah. I would I would agree with that. Yeah. And and and to emphasize what John's saying though, it's about reducing the friction in that process from the stakeholders point of view.

You know, if you've got finance going to John saying, hey. I need an asset inventory. And he's like, great. Check back with me in a month versus, you know, yeah. Sure.

Click a button, and here it is, you know, on demand whenever you want it. That level of friction has always been part of the network automation narrative. You know, back to John's, you know, I I need to push VLANs. You know, that's always, like, the classic narrative. It's like, you know, how many network engineers does it take to push a VLAN?

You know, it's like so, it's all about that friction. You know, we always talk about how the networking teams were lagging behind the cloud or the server teams. It's all about this friction. And the friction isn't between network engineers and themselves because, you know, most people do whatever. Yeah.

Yeah. The friction is between the network and the stakeholders who want the network to serve some purpose for them. Yeah. But and there is, of course, the story of, right, we need to do things quickly and we, in in ways that we've not been able to before because the demands are higher from from those stakeholders. And I think that from from standing up for the network engineer here, that that that it's always been about, right, we we just don't have the visibility, the understanding of the environment, the documentation, whatever it is, in order to be sure that we can deliver things as quickly and and in the way that we want.

So so I think I think the way that the network automation story has played out from a high level, I. E. Get things automated configuration wise, get that visibility of for for the network engineer has has had to happen because you've had to get network engineers into a position where they understand what the automation can bring them. But, obviously, the story is much more than that. Exactly the things that you've been saying that that we we can get things so far that way, but, ultimately, this is about making sure that everybody who consumes that network in some shape or form has access to, an application an application team building a new application and need to understand what, you know, the the the network will carry that application and that the security is set up the right way and blah blah blah.

Or if it's the procurement folk who are looking after, you know, maintaining those service contracts you talked about before, John. You know, it's whoever needs that service, we need to be able to provide them that service. And I think it feels like it feels like in the last the last couple of years, things have have shifted from this idea of networking being the the cables and the devices and the stuff that makes packets move from one place to another, and resilience and high availability being that that I can fail one link to another. To to now becoming a story of we're providing service and we're making sure that that service is available. And this feels like it's been driven partly by cloud, I guess, and the way that the cloud approaches have come along and and meant that you've got to have that higher level view.

But, you know, it also I I guess this is why we're pushing into thing you know, some of new operational areas. You you've both well, all of you have mentioned chat ops in different ways, for example. And I think it's really interesting that we're looking at different ways of interacting with this data based on the fact that we're appealing to different folk. Right? Well, that's the that's the problem I have.

If I if I'm talking as a consultant, I'm going to different companies. And if I'm talking to a technical, person within company x, you know, I I say, oh, we can easily find out. Let's take the VLANs. We can easily see what VLANs are configured in your network. They go, so can I?

I can run a show command. But it's trying to talk up the line as, okay, if someone else wants that data, we need to be able to present it to them and make it consumable for all rather than just yourself because it it's that translation of data, and I think that's the important part, as you said, Darren. Yeah. So I wanna I wanna actually, you know, focus on what you just said because a lot of people gloss over the translation of data. Right?

I know what a VLAN ID is. Yep. I know what the VLAN ID 275 means in the context of this building. Mhmm. The stakeholders in my network doesn't care.

They're like, I need to add this printer to the network on floor 5. Like, they you know, so we operate in all of this context rich interrelated distributed numbering goop. Right? Yep. And, and they don't care.

They're like Nope. You know, they're just like, hey. My my phone at my desk isn't working. You know, can you reboot it? And somebody in chat ops just wants to say, bounce phone, you know, either the phone number or the wall jack Yeah.

In chat ops. And and it doesn't matter whether that phone is connected to a Cisco or an Arista or whomever. They just want it done. And so if if chat ops provides that feature for our IT group, then that IT group never has to ask the networking team ever again about it. And, really, the end user is the one who gets experience like, oh, you know, my phone works now.

You know? Thanks. And even allow the user, you know, the desk user to even bounce their own phone. So that's what I'm talking about. Like, nobody needs to know what VLAN it is.

Nobody needs to know what Ethernet port number it is. Nobody knows know what switch it is. It's it's really when when we talk about removing the the complexity of of the of the magic numbers Yeah. You but the the complexity there is is the metadata that and how you translate that requirement into an intent, and that's that's the difficult part. Like, when you said, can you bounce port 5, you know, can we add a printer to floor so and so?

You need to translate that data. And, 1, where do you store that data? And, 2, how do you determine if that's correct or not? This is this is building a consumption. You're talking about building a consumption model for the for for the service, aren't you?

Yeah. What you're saying is this is what it right. What you've got is a bunch of network engineering folks who understand what it means to to find the phone in the network and bounce the port that it's connected to regardless of which vendor it is. So they have to come up with a workflow to make that happen, but then you translate that to be to the to the end user, I need to bounce my phone. So you you set my little self-service portal up and away they go, and they can they can use the knowledge that's in the network engineer's head who's put that that automation workflow, whatever it is together, but they consume it in a simple point and click interface on on a web page or whatever it is.

But it it that that's the point. Right? What you're doing is you're building that you're abstracting the service and you're building the con the consumption model. But ultimately, you're saving the network engineers time as well because they don't have to go do it. Right?

They don't have to I think you're looking I'm I'm look I think you're looking at that problem, you know, in in a way that I would not focus on. I mean, I would I would submit to you that the value is to the user who wants to use their phone. They don't care, you know, if the network engineer is, like, happy or sad. Like, they're just like, my phone doesn't work. I need to make an important call.

You know? I need somebody to reboot it. Is it gonna take me 5 minutes, 30 seconds, or, you know, open up 3 ServiceNow tickets to figure it out? And, you know, then well, I know which one our our users want. And to to Alex's kind of pseudo question, it's like, sure.

It's hard, but I would submit to you that that's the value or the business case for doing automation in the first place. It's like, if if it isn't if it, you know, I'm not saying it has to be hard to be valuable, but most things that are valuable, you know, to a business are hard Yeah. You know, to implement. I I agree with you completely. But but what what what I'm what I'm thinking is that by by by saving the network engineer's time and whatever, they're in a position to actually go build more of these this capability.

And so to to you know, it's a it's a it's a nice closed loop. Right? You you give the users what they want to speed up what they do. It means that there's less pressure then on the network engineers to do what they're doing because they've automated that process. So they've got time to go automate other things.

The key point is that they automate things that that the users actually want to do. Right? And that's that's where the focus has to be. And I guess that's the point, Jeremy, that that, ultimately, understand your stakeholders. Right?

Well, I mean, to Joe's point, when we talk about, like, you know, paying for things, you know, applying money to a problem. You know, if you've I'm sure that there is not well, I don't wanna say I'm sure, but I would find it unlikely if I went to go talk to, you know, other other customers to say, is there a CTO directive that says do more network automation? Probably not. You know, is there a budget for do more network automation? Probably not.

You know, there's probably some business directive that, you know, could autumn network automation be applicable? Sure. Probably in most cases, I would argue. But, you know, that's how you have to kinda look at it. You know?

Yeah. It's finding the use cases. Right? We we we see this one a lot with, security being a great one. Everyone goes, oh, I've got this big security project to make my network more secure.

Well, okay. Go ahead and find out what you mean by make it more secure, and then look at how you can use the data and the the process in order to make it more secure. And there, that's where you have to invest in your tooling and your time to to do it. Right? Yeah.

I just wanna add. I I absolutely love, what these guys are doing with chat ops. I I've dabbled in it, and I have it working, and it's, like, really neat stuff. But I I think this idea of giving your network, right, the big n, capital n network, the enterprise, it you're turning it into a a a member of your team. Like, it's a digital entity.

And we all live in these channels now. Right? The the email box is dead. Phone calls are more or less dead. We kinda live in these little channels in Webex or Teams or Zoom or whatever.

Well, now you have right? I don't know. Nathan, the network, is now a digital entity sitting in these channels with you in in a bidirectional sense. There's something that right? I wanna know if that router goes offline, or I wanna know if, there's jitter about right?

Whatever these things that, to Jeremy's point, the goop that that the people on this call understand, but to turn it into something that business can use. Right? How did you know that the call quality was poor while the conference was going on? Like, they they look at you like this mysterious magician. Right?

That's that's where Merlin and the magic and because you just know the network literally pops up and says, there's some congestion on this link. Is this a problem to you? Do you want me to take some action? I I just think it's so incredible that we're giving a voice to the network and, from every aspect. Like, I have these security triggers in certain high area zones.

If they've if their cert fails and they fall back to MAB, go let the security operators know that there's, you know, a naughty player in the wrong area of the network. Right? Physically located in some in some highly restricted zone, and we just build up that logic around it. And I think, Alex, I I wish we had maybe mentioned the word intent sooner. I think Alex is the first one to say it.

Like, when people ask like, I have a very, specific meaning of network automation in my mind. Like, to me, it means to have a data model, which is your intent that couples with Jinja two templates, in my case, that result in your configuration or the message you wanna send a chatbot or whatever it is, which moves me into a level of almost GitOps. Right? So I make a branch and get I add my 4 VLANs. We do our QA.

It runs through its tests. I do a successful PR into my get main branch. That pull request then kicks off a docker container image build, which then gets released either on demand or at a schedule or a change approved window. And what have I done? I've updated my intent.

That's all I've done. I've added 4 VLANs in a YAML file, and literally every other step in that former CLI login type this type it's all done. I I just merge my VLANs into YAML, and literally the whole rest of the workflow. And and this is Ansible. This is Python.

This is whatever you want, can translate in this kind of intent template push. Right? So, John, I I would submit to you that you've solved half of the intent problem. Right. Because because you can say, well, I pushed a bunch of VLANs that I that I know I should push.

But the real intent is that you have some kind of service operating in a healthy way, which, is more than just pushing configs. I mean, we would all agree that it's looking at the operational state around what that service means, which partially means those VLANs. Right? But it also implies the health of the interface that are using that VLANs. It can also imply, you know, routing, you know, to the IRP.

That's how I mean, it's Right. Yeah. Etcetera, etcetera. And so I, I always get triggered, when somebody says the word intent and they talk exclusively about configuration because at the beginning, we talked about, you know, how that's Yes. That's a that's a no no concept.

Well, Jeremy Jeremy's not setting me up in any way, but but it's a good segue. So as part of that pipeline I described, I should have maybe mentioned this, but, yes, I have a pre state test suite run. I'm using PI ATS and some other tools. Right? So that CICD kicks in.

And to Jeremy's point, he's pushing 4 VLANs, but that means that an IP camera should start working once they're configured or some business impact. Right? Well, I have my PI ATS run pre. I push, and then I run them again post. And then any other customized test suites that I want that give me a bullion pass fail.

Oh, you can now ping that security camera, or that security camera is now sending, you know, the packet counters are going up on that interface or whatever situational tests you want. I I completely agree with Jeremy that that configuration piece I described is one component of a larger CICD pipeline. Yeah? Yeah. But I I would also submit.

Let you get a word in here, Edwin. I'll say that. But I would also submit that that okay. So now you're 3 quarters bathroom, not the bathroom. Right?

Because really, it it doesn't begin and end at your action at the end of your pipeline run. Right? Because gotta say. I'm I'm going So, you know, now you're like, okay. I've added these 5 interfaces to this VLAN.

So now I need to monitor those 5 v those 5 interfaces, like you said, not just once, not at the end of your CID, but it's, like, continuously. And you might need to enrich your monitoring system with information about that interface. So that interface may be connected to, like, you said, like, a video camera, which may have certain characteristics that you know in your intent that, like, that feed should be a a 5 megabit feed or that should be a 10 megabit feed because it's and then knowing that and then looking at, say, flow data and comparing that, I'm like, then that's where you get into this kinda closed loop intent based monitoring or or intent based life cycle, you know, service management. Now I'm gonna shut up. So Alex can No.

I was gonna say the the the same thing. It does it doesn't end once your pipeline is finished. You've still got especially in in my industry, and and we offer managed services. If we've deployed something, we need to carry on monitoring that. Else, we're tied up contractually to to deliver something.

So we need to make sure that it's always up, all the time at to what with whatever parameters. I was gonna I was gonna go for the sales pitch, but I'm sorry. I've I've I've I've I've promised I wouldn't do that. But, obviously, working with IP fabric, that's what what we do is we we do that assurance piece, not necessarily around the the the sort of monitoring the packet counters or whatever, but looking at the health of the of the network as a whole and understanding how that contributes to the the the service that you're providing. I suppose, Jeremy, ultimately, this is this is the confusion of the word intent sometimes, right, around around this because what we talk about is network engineers might talk about intent as being configuration and state.

Application owners might talk about the the intent being in terms of the services that they're delivering. The business are gonna talk about intent as being is there is there money coming in or not? And why is it not coming in? And so it is it's it's having the understanding of the intent all those layers, isn't it? Yeah.

So well, I mean, I I would submit to you that it it's all about the stakeholder. Right? And in some cases, the stakeholders might be the network engineering team, and that's perfectly valid. There's no there's no wrong answer here. It's it's all about the stakeholder and the impact.

And if you and if you can't write down, these are the stakeholders and this is the impact, then you gotta ask yourself, you know, is this really something valuable to the business? And that might be a chain of stakeholders. Right? Because it might be the the the the the Yeah. The ultimate stakeholder might need another stakeholder to be doing things the way that they need it.

They may have a requirement on someone else and so on. So it is understanding. Coming from from my network design background, this is this is, talking you're talking business capabilities and requirements from the top down and working those into technical requirements and capabilities and through. So it's it's a very natural kind of kind of approach. But but the problem is that we're so siloed in organizations that it's so difficult to pull that together.

Right? No. No. It isn't. No.

It isn't. Okay. Traditionally, it has been. Has it has it been? Because because you because people like to say top down or bottom up, but I always say it's it's across the the the horizontal lines of your organization.

So, like, I can easily go and I'm you know, this is my business, but I can easily go to the network operations group, and they talk to the other operations group. And just like Johnny's like, well, we talked to finance. It's it's not top down. It's it's really it's like when I think top down, it's like CEO, VP of engineering, top down, you know, directed. So maybe I'm misinterpreting what you're saying.

No. No. No. I I no. I think I think I I understand what you're saying.

I think the point is that that, traditionally from a from a from a technical standpoint, if if you start start as a network engineer and then grow to be, grow up to be a network designer and move on and to to become, a network architect, you've got a you've still got a very narrow view. You've you've you've widening it as you go because you're understanding more about the things that are going on around. But ultimately, you're you're in one particular technical field. And I think this is something that that certainly when when I worked as a consultant and while I was looking at at some of the the the more, complex design scenarios, you had to step outside that and understand left and right what was going on. Now, Jeremy, your perspective is gonna be slightly different because you've been doing this automation stuff in the way that you've been doing it for the last 10 years.

Right? Where the rest of us are just catching up. So so I think what you're used to, that idea of of of going to the right stakeholders in order to get the, get the requirements as you need them in order to deliver the service. And I think this is a mindset change. This is what this is because network engineers don't need to know just know how networks work anymore.

They need to understand about all the other services that have been delivered across that network in order to make sure that they're actually ticking the right boxes in order to deliver the service to whoever their stakeholder is. And I think that's that's part of what's it's it's it's been missing from from network design way back and started getting better, but now we're we're taking this to the next level now that we're talking about operations that are built in the way that we're talking. I think I have a lot to rebut there, but I'm gonna stay quiet because otherwise I I I just think when you go into different organizations, there's a lot of red tape that needs cut in, and and that's the difficult part as being a consultant. You have to get people involved in business process and and things like that involved, and it's not it's not as easy as you would think. So anyone undertaking the journey of network automation, it does take time, and you have to chip away at it with a very small chisel.

But you just said something that as a consultant, and that's that's part of the problem is that that I don't believe a lot of folk actually have a consultative approach that you need to be, to to do the consulting. Right? And that's that that's the problem is people don't know how to ask the questions. Completely. And people don't have experience elsewhere on the other side of the field, of of what happens in another place.

You talk to one service provider, and they have 50 networks. It's not one network. It's 50 networks. So to talk to another, another box, they've got to go through change process, request of information, and and so forth. Go up the chain, whereas, you know, an API would be handy.

They can get what information they want. But but there's a lot of red tape to get that other network within the same business to provide that same information to, you know, automate BGP pairings between them or something. You know? There's there's a it's it's so much red tape. Yeah.

It's a it's a great point. I, maybe a cautionary tale if you wanna look at me and my career and how I've approached this. Right? You don't wanna be painted as an outlaw automator. Right?

Like, as a shadow IT operations. I've sort of been painted without a little bit. Like, you've automated what? How do we do that now? What do you mean they just type in enter and it just does it?

So you you you wanna communicate across the board and really be careful with your processes and integrating. Right? You can't be an island that that is developing solutions all by themselves that people aren't picking up. And and on the other point I wanna make, Darren brought up a good point about tools and capabilities and that natural progression. It it's become almost overnight a very difficult job.

The network engineer, the network designer, the network architect. It's not just the CLI anymore. It's not just figuring out configuration stanzas. I think in this hour, we've probably talked about 35 different tools already, Docker pipelines, PyATS, Python, Ansible, on and on and on. But but use this for inspiration.

This is more money. This is more, prestige. This is more respect. This is a better career path for you. If you've learned Ansible and Python and pipelines and Docker and Kubernetes, and there's they're valuable skills.

And you may feel like I'm doing this in isolation. Like, am I am I an anomaly? Is anyone else even doing like, I've partly, I wrote the book to send a throw a message in a bottle out to the world. Does anybody else doing automation this way? Because I feel like I'm I'm, like I said, a automator, but but people are doing things this way.

The rest API means something today. The tools are there to really go out and make make a a difference in the world of technology today. You're not just typing in, right, VLAN 10, name blue, enter. You're you're you're solving problems using very high functions in your brain. And what I think is neat is that you can write things down in human language.

K. 1st, I wanna capture the state that I wanna push a config then I wanna recap. Right? Draw it all down in human language, and then it's a matter of translating it just like you would do Google English to French. Well, how do I take this intention of mine in human English and turn it into an answerable set of tasks or a Python series of commands?

Right? That's that's all you have to do is find that common dialect between your intent and how to do these things programmatically. So so one of one of the one of the so I'm gonna, like, again, cautionary tale because, I have I have a a road of broken glass of them for sure. So one of my inherent biases that was a cautionary tale was, as Darren pointed out. You know, I have a software engineering background even though I've been in networking for a long time.

So I have an inherent bias of how things should should work, how how to structure a program, how to decompose a problem. Should I use a list or a dictionary, should I use Ansible, should I use PyETF? Like like, there are things that I don't have to think about because it's just I've been doing for 20 years. Yeah. Just like a a network engineer can probably subnet a network in their brain, and I'd be, like, drawing out that for, like, an hour and have, like, spreadsheets.

I'm like, how do you guys do this? It's magic to me. So what I found is is there are some very basic concepts that enable software automation that most people don't have. And we and everybody glosses over. Like, honestly, it's like, oh, well, you just do this, and you just write a playbook, and you you know, it's like and and I'll talk to somebody, and they're like, what's an environment variable?

Right? And I would legitimately, like, you know, like, there are people who've done you know, they come from a Windows world, and they've never done anything like this whatsoever. And I and and when I talk to those people, they feel crushed by the pressure of what, you know, we're doing. You know? So let's all acknowledge that we are way on the bleeding edge, the tip of the spear of network automation.

The things that we're talking about here are way super advanced kind of things. You know, they're meant to get people to think about, you know, what is the art of the possible, or like you said, inspire people to do things. But when I start people on, you know, automation, I like to go back to what you said, John. Start with a spreadsheet. Like, automate a spreadsheet.

I mean, legitimately use a spreadsheet as input. Like, say, your users are gonna give me a spreadsheet that says make these port changes. Okay. Like, can you programmatically get data out of a spreadsheet and then maybe template build the configuration? Don't don't try to push it.

Don't try to do match with pipeline. Like Yeah. Just prepare. Yeah. But but these basic steps or these I don't wanna say basic, but core concepts that we take for granted because, John, you have a software engineering background or programming background, Alex.

You have a programming background, you know, I I believe. Or you fly a plane, so that's close enough. You know? You know, we take these things for granted, and it should not be lost on people like us who who talk a lot in in a public forum that there's a lot of people who are confused or or aggrieved by people like us who talk about, oh, this is easy, or it's just a REST API. And it's just like they just go crazy with they feel like they wanna throw daggers at us Right.

You know, metaphorically, I hope. Well, I I find a similar one. We're not delivering training courses, whether it's Python, Ansible, or what what have you. I find that a lot of the struggle is not necessarily the tools we're teaching. It's it's just a different platform.

They're used to a CLI where they can just tab, you know, start three letters in tab and and, you know, and get started, whereas it's it's not the same mindset in the in the same space. So I think I think that's where people can trip up a bit as well in Linux or whatever platform you use. Yeah. You just said it, Alex. That's and and Jeremy on this, inspirational topic.

I I completely agree. I don't want anyone to think that that I believe it's easy. It it's become easier for me. Yes. But but it's like any other skill.

If I'd been playing piano for 4 years and you sat down at the piano beside me, yes, I'm going to be better at piano than you because I've had 4 years of experience doing it. But I think the biggest barrier I think the honestly, what what probably throws 75% of people when they try to approach this is Linux. Right? We we come from a Windows world. Right?

The Linux desktop never materialized. And and up until recently, it's been tough to, what, am I gonna put VMware workstation on and spin up a little Linux VM to learn it, or do I go get a an old box out of my closet and spin up a boon to on it? How do like, it was hard to break through that. But now I think you know, I'm not trying to be a Microsoft evangelist here, but Windows subsystem for Linux and the Linux image of your choice from the Windows Store, a few clicks now, you can have that Ubuntu environment and and make a directory. Do LS.

Right? Start doing basic basic day 1 Linux things because I'm telling you, Ansible is not coming to Windows. Py ATS and Python, they're not coming to Windows. No. They're giving you a portal through WSL.

Yes. But don't please please don't walk away and go, like, I'm never gonna do this because a Linux thing, and I don't know Linux. Maybe make that your first thing to to do for the rest of 2021. Make a folder, do some directories, remove a file, make a file. Very basic, and don't think that they're below you if you don't know how to do those things.

I mean, I happen to grow on DOS, which is an easy parallel over to Linux. And I've, you know, I've used Linux for quite some time, so I'm very comfortable with it. That I I again, I maybe am a bit of a unicorn. Not in every network engineer even right? They don't even know where to start with Linux.

So I think that's a big barrier for for entry right now, along with REST and JSON and more programmatic concepts. My my one of the the first suggestions I give folks, is go get yourself a real IDE like PyCharm or Versus Code or something. You know, I I I personally use, you know, PyCharm Pro, and I've been using it for, you know, a very long time. And what most people are going to start from without that is, oh, v I, if they're in a Linux, or notepad, you know, if they're not. And I've listened to countless horror stories about people using notepad and writing Ansible playbooks and copying them back and forth, John.

You know? I mean, like, seriously. Guilty. Totally guilty. I know.

I know. This is your story you told me. So but my point is this. It's like, if if somebody sat down and they said, okay. We're gonna begin your Python journey and start with PyCharm, for example.

And they they saw how an how that environment shows them where they've typed something wrong immediately. Or you can tab complete to get a function name or something like that. It would, I believe, reduce the friction or the the anxiety Yeah. About, gosh, you know, I'm trying to write code and it's overwhelming because their their their friction point of view is notepad, which I would shoot myself, right, in the face or VI, which I would just scratch my eyes out. You know?

But no. But I've used both of those tools in my past, but I would not try to attempt even writing Ansible playbooks, right, without an IDE that will automatically fix my indentation or automatically tell me where an indentation is off by one second. Like, these tools reduce the friction and the anxiety of, am I doing something right or wrong? And I wish more people would start there than, oh, let's, you know, write an Ansible playbook. It's like, okay.

Before you write an Ansible playbook, make sure you have something that will look at your YAML and make sure that it's okay. Yeah. I I I completely agree with Jeremy. He you should listen to this advice. And I think, right, it it sometime maybe it's the what we talk about.

We talk about network automation. K. I gotta write a playbook. Let's just pull it back a layer and talk about infrastructure as code. Well, code Jeremy's point, he's done it for 20 years.

There's a certain way, a certain set of practices around code. Right? Version and source control. Without Git, you're not going to go very far. Right?

Are you gonna TFTP that file back and forth like I did? Are you gonna have working working underscore version 2 version new should work dot neat time. Right. Like, you can even have a story. It's a great story.

Right? I was I was copying and pasting stuff out on notepad, so I've got no no way to talk to about that. But But yeah. So, yeah, so start working with code appropriately. I need version of source control.

I need Git, which means I need to Git repo, which means I need an IDE, which means I need linters and extensions and other tools, and it it becomes so much easier. I I can remember the first time I opened up a YAML file in Versus code, and I went, wait. Like, it tells me if I have too many spaces? Holy cow. Holy cow.

Is that ever incredible. Right? Yeah. So, like, all the things that you just described, you know, like, a tool like PyCharm, just you click a button, it does all that automatically for you. You don't have to sit there and install every little bit or in every single tool.

It's like, we talk about a virtual environment, and people are like, what is that? Literally, when you click that open new project in in PyTorch, it automatically builds all these things for you. It sets it all up. It reduces the friction. I'm sure Versus Code does the same.

You know, and and it really does drop the barrier. And I don't I have not really seen too many, videos or focus on this other than from the vendors themselves because most most people are like, meh. You know, it's like, you know, people who've been doing software engineering, they're like, yeah. Yeah. Sure.

Sure. Yeah. We know this. Okay. Right.

And, like, if you're coming from the outside of software engineering and you're trying to just get started with network automation, you're like, what is this instead of land of unicorns and clowns? You know? Like, what is this? And it's and it's overwhelming unless you unless you have somebody to shepherd them through that process. Yeah.

Yeah. One for me was, I was teaching an Ansible course, and, I think I set up all the v I I set it up so that Versus code is on all the box is when when it comes and I'll say that's how we do it. But, like, control s, when it saves, get to automatically format your playbooks. And it does like, oh my god. This is another level of, my spaces are not wrong now.

Yeah. Or there isn't a tab in there. A lot of people are getting frustrated of tabs instead of spaces. I said, nope. Just control f it, and, it'll format it fine.

Yeah. Nice. That's ticking right down to time. If I was to put you on the spot, anyone want to take a closing comment for the last 60 seconds? Or I I wanna hear Alex, like, give the advertisement for his, you know, natural language processing chat ops and just talk about, like, the way like, I think Alex is on the way bleeding edge, so I really want, like, the future the future of the future.

Go on. Go on, Alex. What is it you It'll probably take longer than a minute, though. It's okay. So, I set up a requirement I had was to reduce power consumption.

In reducing power consumption, we 1, we the the goals of the company. I was at a conference, and I I there was a big questionnaire led by AWS, and I won the questionnaire. It was, like, kind of shout out the answers and you win it. I won it. I won a Amazon echo, and someone in the business said, I bet you however much.

You can't make a bot to basically ask your network of using Alexa. So I said, okay. Sure. Next week. I said, here you go.

You can collect what's so and so in the network, and it would respond with the data. So kept went down a huge rabbit hole of utterances, intents, all this type of mad stuff of being able to describe what you want to a voice platform and be able to get an an intent out. After doing all of that and being able to say, you know, what is the power usage in so and so rack? And I'll go, oh, it's 1,500 watts in so and so rack. And then I realized that if you want to actually query the data using voice and using a human voice, it's actually very hard because as Jeremy said earlier, we have hostnames for devices that do not make sense.

They determine what build in a locate devices, what services could probably be on that platform. So saying, hello. Can you get the information for b 161976 a l does not make sense. So we we found that voice is probably not the best way to go and type in it, and that's where me and me and Jeremy have been communicating on the the Slack and the teams and so forth, but we're working on kind of, an algorithm that will, you you'll be able to type a question, and it will determine the intent of your question about the network using IP fabric in the background to get the information. So whether you want to configure something or get something, it will, figure out your intent dynamically and and do that.

I mean, I I guess what that's doing is is answering Jeremy's original point about who's the stakeholder for all this. Right? And and and It's not the network engineer. Exactly. If a network engineer wants his his stuff, there's no point in going through No.

Through a voice interface. Right? But but that said, maybe, you know, if there's something if someone wants to turn around and ask Alexa, you know, is there a is there a problem on the network with me trying to get to the the application I'm trying to do? And it says yes or no. And I've and I've and I've told your your network team about the problem that you've got, then all the better.

Right? And it all stems down to what Jeremy said earlier. You don't want to ask the question, what is so and so like, b 1619. You're asking, is this service healthy? Is so and so healthy?

And you need to do that translation in the background and determine the intent based on the the utterance that a a human has said. And it's that mapping of what someone has said and translating it to the actual intent that's actually the really hard bit, and that's quite interesting to research. So does does my IP camera work? Yeah. Well, what IP cameras?

What do you want to test on those IP cameras? Is RTSP working? Is, you know, can I ping it? Yes. But the actual protocol is not running over that link, so it it's not working.

There's there's a lot of yeah. It's gay gets big rabbit hole, big rabbit hole. So I'm I'm gonna have to drop for another session, but what I would say is if you guys wanna wrap up and let people know how they can follow you and get involved in more of the conversation, I think that'd be a good way for you to round up the conversation. So I'd like to thank you all for your for the audience, for the delegates as well. It's been a a pleasure to be involved, and, good evening, good afternoon, wherever you are.

Thank you. Thank you. Cheers. Thanks very much. And there we go.

Yep. So you guys can find me on Twitter, at networkautomaniac. That's NWKautomaniac. My best way to contact me is through either one of the many automation Slack groups that are arranged. Just search my name, Alex Giddings.

Twitter, mini trigger. It's a bit of an odd one, but, mini trigger. But I'm not very active on there, but I will respond. If if not LinkedIn, just search me or or GitHub. Yeah.

Hey. Yeah. I'm on Twitter, john_capobianco. I'm very active on Twitter, and maybe too active. And, I can be found on LinkedIn.

I'm very open. My, automate your network is my GitHub repo. There's Ansible stuff there. There's my new Merlin on there. If you wanna get involved, like, if this is something you wanna sink your teeth into, I mean, I need help with HTML.

I need help with CSS. I need Python. I need everything. Jinja 2 templating. You know, connect with me, and I will help you get going.

And, you know, I I love these discussions, and thank you for inviting me. Thanks so much. Yep. Thank you. Thank you, Darren.

Yeah. It's an absolute pleasure. Thanks for thanks for joining us guys. It's been really good to get involved.

Podcast notes

Episode Title:

Next-Generation Network Operations

Hosts:

Joe Kershaw and Daren Fulwell

Topics:

  • Network Automation
  • Network Operations

Our hosts