Read blog
De-risk your SD-WAN rollout with network digital twin technology.
read more

Automation State of the Nation

IP Fabric Chief Evangelist Daren Fulwell and NetBox Labs Chief Product Officer Mark Coleman are here to take a thorough look at the current state of Network Automation. What have we learned so far on this journey toward automation, and how will past and current experiences shape where we go next? Are there still gaps to cover and room for improvement? Tune in to find out what these seasoned veterans of the automation space have to say!

Transcript

Hello, and welcome to another episode of the Community Fabric Podcast where we bring the networking community to the table to talk about the things that matter to them most in their day to day. I'm Darren Forwell, your host for today's conversation where we're going to take pause for a moment. We're going to look at how things are going in the world of network automation, consider what we've learned so far and how it shapes what we do next and we're joined by someone who's in a great position to help us with this, being part of an organization which has received a rather large vote of confidence, recently. Is that fair to say, Mark? I think it's a pretty big voter confidence.

Yeah. I'd absolutely agree with that, Derek. Would you like to introduce yourself? Absolutely. My name is Mark Coleman.

I'm the chief product officer for, NetBox Labs, and I'm based in London, England. The center of the universe, of course. But, it's amazing how many how many British people I end up speaking to on the podcast. It's, anyone would think there's a bias there. It's Well, it's interesting in the network automation world because there is there seems to be a, like, a non statistically, reasonable number of network automation people in the UK.

I don't know what the history of that is, but there's a lot of us here. That's a very good point, actually. I've not considered that before. You know what? We should we should get them all together sometime maybe.

That's, Like a WhatsApp maybe. What do you reckon? Yeah. So there's a thought there's a thought. There's there's well, watch this space everybody.

So and I mentioned the vote of confidence. Obviously, anyone who follows industry news over recent weeks wouldn't have been able to avoid it. Right? You and the team must be on cloud 9 at the moment. Incredibly happy.

Yeah. It was a lot of work. I mean, the story, obviously, this NetBox story starts a long time ago, but the the sort of run up to this starts in, January 2022, when I came over to NS 1, where Jeremy was at the time, and we decided to incubate users. And it users. And it seems like the answer was yes.

So most of 2022 was ramping that up, experimenting a lot, trying to figure out what it was, and then it became very clear towards the end of the year that this this sort of had its own life. So, so we were keen to to to split that off and and get some funding, which we're able to do successfully. The Friday before last feels like a week. It feels like there's no time and 10 years. Yeah.

Things are quite busy at the moment. Yeah. Yeah. I can imagine. And I guess I guess what it does is validate that the network automation space is generally a good place to be, that there's a lot of interest, not just from from, practitioners, but but investors that that there's something real in what we're trying to achieve here.

I think that's right. I think network automation has had a bit of a rough rider's term just because, you know, some industry commentators say, well, we've been doing this for a decade already. But really, what they're referring to is people were sort of calling SD WAN network automation a long time ago. And I think what we're really seeing now, and that's where, you know, people in the industry are watching and investors, is, is network automation proper, as I would call it. Right.

It's starting to cross the chasm. People are understanding, you know, this is on lists, you know, CIO lists for the for the year of things to be investigated. And I think also the community, is responding by creating all sorts of incredible open source tools and closed source tools, and and you can see a real buzz occurring here. And people are starting to pop out of the woodwork with real success stories about how those help their businesses. And when you bring all those things together, I think that's when you start to see the melting pot sort of, hotting up.

Yeah I mean it's felt felt to me like, a lot of the drivers obviously being at the ground level right, the people gearing up trying to understand what approaches to take and and looking at projects that involve I guess, the the the obvious things of being able to to push config, to be able to gather data and so on, and and and use those things then, at the simple level. Now what we're talking about is stepping it up a notch, I suppose. I think that's right. I was I was doing I was having some fun at the weekend doing a deep dive on monitoring, and obviously trying out the fantastic new IP fabric free trial. Yeah.

And, and and it occurred to me that, you know, there's a lot of action in this space, but I think exactly to your point, really, really, if we think about what we want to do with network automation, it's it's it's to just do the thing we were already doing, but with less errors and a greater scale and more reliability and etcetera. And what that basically is is I would like to express my intent to the network, and I would like to the network to give me information back about how it's working in a way that I can understand. Yeah. And I think, you know, with the config pushing and and simple monitoring of things, people have definitely conquered that already because it's quite logical for them. I think the next step is for us to figure out how to make, you know, to be able to get to a point where you can reason about what you would like the network to do at a higher level than just devices.

I I think we need to do a bit more work around the tooling. And I was just taking a look at this. I was like, is there a stack here? And and I started right down the bottom with the devices. And then I was like and then you've got all these libraries like PyAPI and Juno's easy NC and all you have a 1,000,000,000 tools down there.

And then you've got SUSYQ and Napalm, and then you've got Ansible and Nournia and Terraform. And then you've got you know, only then do you get to, like, IP Fabric, NetBox, and then so combining all those, especially when the abstractions are a bit leaky, you've got you don't have skills in all of them. Yeah. The tool choices you make have sometimes unforeseen consequences to future decisions you might be able to make. So there's a lot of work we can do here to help that that network automation journey be a lot more intuitive.

And, is this something that you see, NetBox Labs? Obviously, you're you're going down the path of, of creating the consumable, Netbox cloud instances and so on, but obviously there's more to it than that. Right? Just just giving people tools isn't enough. That's right.

Yeah. I mean, really, what what what we're saying is that there are sort of 2 big problems, data models and workflows. And, you know, I mentioned all these tools earlier. If I pick, for example, Libra NMS, if you pick LibraNMS, you automatically get napalm, and you automatically get a certain data model. Now a lot of people are successfully using this with with with NetBox, for example, but you have to reconcile that data model somehow.

So so there's a lot like that in there. You know, you we've been working on a on a, plugin, and Alex has done some great work there to figure out, okay, how do I translate IP fabric world to to NetBox world? And and it's not that having the different data models is necessarily an issue. It's just an issue if we don't recognize it and make it easy for the for the customer to or for the end user to to reason about. So that's the data model side.

I think the workflow side, what that we're thinking about a lot is kind of related to the the point that I made earlier about there being all these tools with different abstractions and everything else, and and it should be kind of irrelevant. Like, right, like, what I want is, like, BGP to work or x to work. And what we're making people have to peer through the through the, you know, through the abstractions way too much right now. Yeah. Yeah.

And and there's kind of a there's a trick in there, but I can talk more about how we're Yeah. About that place. It's it's interesting because we, you know, as as IP Fabric, our our our our the key aspect under the hood, and the people people probably don't see this necessarily, is actually the modeling of the behavior of the network. Yes. There's a a shed load of data there that's gathered.

Yes. You can do interesting things with that data. But unless it's structured and unless it's normalized and and all of those good things, you can't compare one thing with another. You've always got to go through a process. So do this for that vendor, this for that vendor, this for this API, that for that CLI or whatever.

Yep. And and I guess this is the point, isn't it? This is it's about being able to create that that view of the network as a whole without having to dig into the vendor specifics or the or or whatever of the individual technologies that are deployed. Mhmm. Because that's just you know, that way, you know, that that it sounds dangerous to me.

You know? It's it's it's a hideously difficult and complex place to, to find yourself. I I think there's a balance to be found, and it's very difficult. I was fortunate enough to be quite close to the I didn't never worked there, but I I worked with HashiCorp a little bit back in 2014, 2015, and I was always inspired by the way that that they looked at this, which was, you know, it was a huge space, which we called DevOps, and then we called it Programmable Infrastructure. Now it's called Cloud Native.

But that space that they were looking at, You know, what they've what they were very successful with was releasing a set of tools that were individually useful, but when combined, more useful. Yeah. And I think this scale we have right now and I can understand, you know, CIOs pulling their hair out and saying, well, this all just looks really complex. Why don't I just buy the entire stack from 1 vendor? And, you know, that's perfectly reasonable logic.

Right? It's like it'll be easier if it all worked together, but But I think people have already been burned on that a little bit, and they get a bit worried about vendor lock in being tied into a specific road map. What if I can't integrate it with something else in the future? How much influence do I have? And that's where the appeal of composable open source comes in.

Yeah. But then you've got, well, I've got to glue it all together. Yeah. And I think what we're looking at is how do we have an open and composable approach to tooling that enables these workflows that is opinionated but flexible. Right?

So Docker used to say, batteries included, so it works out of the box, but you can change it if you want to. And finding that balance, obviously, there's a lot of devil in the detail there. Yeah. But that's something that we're very excited about looking into. I I I do think I mean, that's something that we've been looking at quite heavily, the the the ecosystem approach generally because, you know, that that's what APIs are for, right, are are about being able to to to bring together capabilities, disparate capabilities and disparate tooling that that can work together to give a better outcome.

And I think that been super important for especially for for something like us where where it's about making that data available and creating inferences from that data, which are then making those available as well. Right. To to enrich the the the overall operational experience. And I think it's it's interesting. I obviously, I come from a heavily networking background.

I've built networks, supported them, maintained them for for years years years. The tooling has got steadily worse, but that the main reason is because networks have got more complex and we're still using the same tooling we were using back in the nineties, right, so it's it's got harder and harder and harder to to understand how to how to go about this. What we're now talking about is something that understands relationships between devices rather than just individual data points and suddenly, things start to get a bit more interesting again because now we can understand, ah, there's a there's a there's a a common behavior of the system rather than, looking at individual data points from individual devices and expecting to be able to extrapolate behavior from that. And I think that's where the ecosystem comes in. I think that's that's really right.

And a lot of what we're seeing right now, it is, well, actually, we did our fireside chat in Amsterdam. We did. And And I think it was called network consummations DevOps is just like DevOps apart from the hard bits, something like that. I can't remember which one. Yeah.

Yeah. Yeah. Yeah. But the general point was that there were specific problems with regards to networking, and we can talk about some of those. I think they're actually quite interesting.

It's quite different to a server, especially if you consider how we see it these days. But, but, you know, I remember being at, I remember being at VelocityCon in in Barcelona in something like 2014, and there's this fancy new thing, which was a plug in for Nagios. And whenever there was an alert, it could run a hook alert hook to go and pull extra context. And to your point earlier, I think that's what we're starting to see. I was speaking to one of our community members on Friday night, and and he said, you know, when I get an alert, I want an alert with all of the context I need to make a decision, not just like there's an issue with this device.

Because then I'm I have to get into detective mode again, and I think that's about pulling it up to workflows. If my workflow is debugged, then I want tooling that supports me debugging and doesn't require deep forensic work or not too often. Yeah. And and this is where where the hooks into ticketing systems and and that kind of thing gets gets used for, right, because you can you can automate so much of that gathering of data when you've got the data there and available and the understanding of what that data brings to the process. And to to your point, workflow, it's about how you would handle and manage that that kind of incident.

What what happens when it arises? Of course, being that we have to, you know, we have to log all these things and track them and audit them, so we have to go through that process. You know, this this data ends up in all kinds of different systems rather than just necessarily network operation systems. You've got to you've got to have audit. Right?

You've got to have ticketing and those sorts of things. So these become super important too. Yep. But this is the point, isn't it? This is about understanding what we can contribute to that, from from whatever systems we have available to us.

Yeah. Absolutely. And and that's where the idea of information intent comes in. You know, I would like to express my intent to the network in a way that's, like, in language that's or, like, in in a maybe not in the same language, but, like, in the terms whether Yeah. That was in the change ticket that asked for it.

Right? Because, ideally, you'd be able to express business value in there, and then you'd also be able to, to understand at a high level when it comes back in terms that you understand. And I think that's that's a lot of the work that we're we're we're thinking through. But then yeah. And then that's the case of you're you're getting into the realms of almost self-service and that kind of thing here, aren't you, by being able to to generate that level of automation?

That's definitely something that we see happening very often with with NetBox, both open source users and our NetBox cloud customers. A very common use case is you've got a central NOC that's responsible for handling all changes to the network, and it's too much. So what they do is they make sure that everything's in NetBox with, you know, additional tooling around it, and then they use the permissions model to farm some of that out to support desks. So not only bigger things need to actually come through to the core engineering team. If you're changing a VLAN or, you know, changing the status of some device somewhere, you can do that elsewhere, and that helps to push the work to the right people as opposed to it being all, you know, focused on this one team who, as we know, are already busy and overworked and stressed and everything else.

And and there's and there's the key. Right? Is you're you're creating this open data that's available to whoever needs it in whatever form they need it and I was talking with, some of our our BDRs to, earlier today about using the data in in chat systems and that kind of thing to to to crack it open and make it, accessible And I think these these things are sometimes overlooked. Right? You know, these are these are great methods of of communicating that data out to people who who wouldn't want to log in to a platform and go do a thing.

Yep. It comes back to the workflow again, I guess. It it really does. And one of the one of the things that we're mapping on when we're thinking through workflows is for whom, and and, you know, often we'll see, you know, like, risk would really like to know, what version of all platforms are on all devices and how many CDs are we aware of. Right?

And and they should just be able to ask that. Or finance would like to know, you know, the age of all the devices and how long we've had them so we can amortize those on on the accounts. There's a lot of information in the network or the networking equipment that the the the network engineering team doesn't really need to marshal. You know, they should they can just open it up. And and yet, you know, that that the way things happen right now is that they'll get a request from from the the people who look after the support contracts and say, right, vendor x thinks we've got all this stuff.

Is this true? And where is it? And is it physically in the network or is it not? You know? And that sort of that sort of audit can take weeks to manage and handle because of the effort right that's involved.

Yeah. So Mark I mean obviously, we we know what we need or what we think automation can bring, and I guess what we we're trying to do, I suppose, is help people along that path and and and give them the tools to to do that. Do people need a bit of bit more of a nudge or or what more can we do? I think, you know, you can answer that 2 ways. I think there is do the do the tool creators need a bit more of a nudge?

Does the community need a bit more of a nudge to to use them? I I think both are true to some extent. You know, we we touched on bringing the primitives up to a level of, you know, workflow and and intent. I think that that's something that we need to start focusing on now. And I think it's important that while we're doing that, we don't leave behind people who are just getting started with simpler automation workflows.

I think from the community perspective, you know, I'm just seeing a huge amount of excitement. One of the great things about being with NetBox, with this huge open source community is every now and again, a massive multinational company will just email us and say, oh, by the way, we're using this. You know, sometimes, like, a 1000000 IPs and everything else. And and, you know, the feeling that I get is that, sure, it's a little complicated, and there's there's a lot of work to be done there. But in general, people seem to be charging ahead.

And, you know, one one data point to sort of make this a bit more tangible is how is it Nanonc86, maybe? The one in LA, last year. Few talks on network automation, but not a lot. The next one in Atlanta, huge number of talks on network automation, and I think people are starting to see this now. Obviously, Cisco have done a lot of work with the DevNet.

That's that's been huge for a while. Yeah. Yeah. But it it really seems to be popping up. And one of the other things we've noticed, we've we've been we changed the format of the NetBox community meetups a couple of months ago.

So it used to be more of a webinar format with general updates for the community. Now it's more interactive, and we have people coming to talk. Alex came, I believe. Yeah. Yeah.

True. And what we're seeing is a lot of demand for Net NetBox related subjects that touch on automation. So there's definitely hunger for for knowledge. So really then it feels like, examples I guess and and and learning from the people who are doing already, you know, making those examples available, giving people the opportunity to demonstrate and and and show the art of the possible Yeah. Becomes super useful.

I think we can do a lot to help there, and that's certainly something that we're looking forward to doing. You know, I I always think back to one blog I really enjoyed from a company in London called Container Solutions during the whole cloud native thing, and they had a maturity matrix. And it was very useful. You know, it's pretty high level, but it was useful because you could plot yourself on it and figure out where you are and where you should go next. And, you know, the Cloud Native Computing Foundation had their cloud native journey.

And it's not like every journey should be that way, but it just gives you some signposts as to, like, where you might begin, what to do next. And, of course, there are a lot of excellent resources out there. Eric's, Eric Chow's books, very good. Rick Donato's out there with his packet coders. Anton, you know, a lot of people that we know.

Yeah. There's a lot of information, and I think they have the same challenge on the training side, which is, you know, it's got to advance beyond here's a bunch of tools and how you plug them together, And it's got to become these are the business object these are the business outcomes that you achieve with those tools. And I think that's work that we're all trying to do. Yeah. I I mean, that obviously from a from a commercial tooling standpoint, that's something that the IP fabric is is regularly having that conversation.

It's it's where's the value in doing this. Although it's not, you know, an automation tool in the same sense as as, as some of the others you described, the the problems are the same. Right? The the the challenge is, our competition is is the way things have always been done. Right?

That's ultimately it. Why why is what what I've always done not good enough? And sometimes that's not visible because almost network teams are overachieving in in supporting and maintaining the environments we've gotta deal with, using techniques which are which are not necessarily, the best for for modern complex networks. And so it's not seen, in the higher ups that there are issues or or that those there are ways that that that the supportability, the availability of the network can be improved by using these techniques. And I guess this is the this is the balance.

Right? This is this is where we need to have those tools. We need to educate people in using them, but we also need to be able to show what business benefit comes from using them, and and be and and help network engineers educate their own leadership as to as to how these things can help. I think bridging that gap makes sense, yeah, and we I think it was sometime in the middle of last year Rich and I were like, we should write network automation for managers' book, and and that remained a very good idea that didn't get worked on. But we'll probably get back to that at some point.

I think it's sounds like it sounds like a blog waiting to happen there. Then. It should probably start with a blog, and we'll see if we can get a book out eventually. But, but, no, you made a good point about, you know, helping the network engineers. I think we like to joke that NetBox's main competition is is, Excel.

You know I'm sorry. The number of people that come in, you know, and we're like, you're managing how many IPs in Excel? Like, this how does that work? But I think that's actually a credit to network engineering teams because, you know, they they're incredibly disciplined. Yeah.

We've got, you know, strong human processes to make sure this thing doesn't fall apart. But the problem is is that whenever I hear discipline, I think danger because humans are not good at that. And if your process requires, you know, a strict set of things and if someone forgets 1 and then it all breaks, then you're sort of you're sort of pushing that tooling a bit too far and you probably need to look at something else now just so the cognitive load isn't so high. Completely and and again we'll we'll say the same in terms of diagramming, in terms of, design decisions and and, the way networks are put together and the reasoning for that. Same same thing.

So, yeah, absolutely. So, you know, some common themes, coming from that, so that's, that's good good to hear that, that we've got that position. So, so where do we go next? What do we do? How how do we, you know, you you guys have got such a great, established community, I suppose, around NetBox, So many great use cases.

I can't even imagine, where you are with planning what comes next. Well, I mean, you know, our main focus right now remains NetBox Cloud. We want to make it easier for anybody to use NetBox, and and, that applies to existing open source users. But, also, what we're noticing seeing is a a huge signal of people who have never used the open source, and they're coming straight to NetBox Cloud. And that's probably because they either don't have the time or the inclination or the skills to go and run this thing themselves.

So coming straight to Netbox Cloud is very interesting for us, and we're working on that. You know, a lot of running Netbox at scale, we've removed a lot of the operational overhead. You know, in in 2022, our hypothesis was definitely network engineering teams are too busy to be running a piece of software that's so critical. And because often they're not, Sys happens. So, you know, so it's risky.

It takes time. So we wanted to solve that problem first. What we're seeing now is, demand for a few other things. So so one is, you know, when your when your NetBox instance becomes the core of your network automation, you're pulling information, pushing information all the time, You you really start to need things that resemble software development life cycles. Like, you want test environments, and you want to be able to move things across.

And helping the larger users with those kind of problems is something that we're looking at very closely right now. And then one of our other big focuses, which sort of splits 2 ways is just around the ecosystem. You mentioned ecosystem earlier. And one of the things that NetBox really, enjoys is just a huge ecosystem of plugins, and many of them are incredibly well maintained by the community. What we're gonna try to do is to raise the quality of them across the bar by helping the community.

And, obviously, you know, our original I think in my first ever NetBox, like, community meeting, I came on and said, our prime directive is to make a virtuous circle of investment with the open source community. And now that we, you know, we have funding and we're able to expand more, what it means is a lot of the things that we see that we could help with in the community we can now focus on and plug ins is going to be one of them. Yeah. That's cool. That's cool.

I I you know, we've always having those conversations with our customers, they're always talking about NetBox as being the the the way that they want to approach the the whole source of truth thing in their automation. And, and so, you know, it's been all very important to us to make sure that we can we can work in that ecosystem. It's clear as we talk more about about workflow and that sort of thing, but the ecosystem is a lot broader than just, just a couple of tools lashed together. Right? So so it's really good to hear you talk about the, the automation ecosystem more generally.

And I think that that needs to be the way for for all of us working in this space. So I think I was watching a a tech field day video only only today where they were talking about about exactly these these topics of visibility tooling, working with automation and and so on to to give that that full picture operationally. I think it's super important. Yeah. And I think those workflows, you know, our our idea, I think it's in our strategy on our website somewhere, is to be, you know, open compostable network automation, and and we mean that and that with workflows across vendors.

You know, one of the things I've seen done very well by people like Rafana is, you know, they may have a a an integration into another vendor, but it's very well done. And, you know, sometimes what you see is they're kinda glued together and then it's you know, your mileage may vary. But I think when the partners can come together and think about the user to create that so that the flow across tools makes sense, that's when you really start to see extra value being released. So not just passing data for data sent, sake, but but for doing it for a use case and, and understanding that use case properly. Yeah.

Definitely. Listen, Mark. I'm I'm conscious that we could just go on and on and on, and I'm sure we will over a pint sometime soon. But, I don't know if you've got any last kind of little nugget that you want to, to give people and leave them with before we go? Well, no.

I will say one thing, sort of a call to action, is is, we were very inspired by your free trial, and we now have our own. So as we've left you, You can go to netboxlabs.com. There's a big button up at the top, put in your detail, and you'll get a NetBox Cloud instance. There's a whole series of, tutorials that come with that to talk you through it, and it's an excellent way to try it out. So go go go click on it.

Fantastic, well I hope you I hope you see my take up on that and and we'll no doubt be talking a whole lot more about ecosystem anyway as we go but thanks for your time, always good to talk my friend and we will, no doubt see you soon. My pleasure, thanks very much Darren. Cheers, mate.

Podcast notes

Episode Title:

Automation State of the Nation

Hosts:

Mark Coleman & Daren Fulwell

Topics:

  • Network Automation

Our hosts