Chris Dooly (Under Armour) and Lauren Malhoit (IP Fabric) hosted a session at Cisco Live USA to show how IP Fabric can address these persistent challenges in practical, high-impact ways.

In this 20-minute session, you'll learn how to reduce costs by...

• Standardizing data across global environments.
• Validating changes before and after major initiatives.
• Creating custom intent checks to catch problems early.
• Uncovering legacy issues before they cause disruptions.
• Accelerating troubleshooting with end-to-end path analysis.

Watch the full session below, which was originally published by Cisco Live.

Lauren Malhoit: Thank you everyone for being here today. I'm Lauren Malhoit. I'm the Director of Solution Architecture for IP Fabric, and this is my fantastic guest, Chris.

Chris Dooly: I'm Chris Dooley, Senior Lead Network Engineer with Under Armour.

Lauren M.: I think when most people think about Under Armour, they think about performance athletic gear. Chris is is sporting it here today with us. But what they don't think about is the underlying infrastructure that it takes to run your your retail locations, your manufacturing locations, the headquarters—all of those places, and the the networks that have to keep running so that you guys can generate revenue. So that's what we're gonna talk about today.

The interesting thing is that, even though we're talking directly to Under Armour, a lot of this is going to be remarkably similar whether you're in a financial institution or health care system. In the underlying network, we see a lot of the same issues that Chris and his team have dealt with.

And so you've been living the reality. You found a way through a lot of it already, and I know we're gonna talk about the decrease in support tickets and things like that. So getting started, Chris, just tell us a little bit about your role, your team's role, where you guys are located, and your responsibilities and initiatives this year.

Chris D.: So, obviously, I'm with Under Armour. We are a global enterprise. We have engineers across the US, APAC, EMEA—all across the globe. We all have to work concurrently.

My role as a global Senior Lead Network Engineer is I oversee global IT infrastructure. AWS, Azure, Zscaler. Routes, switches, SD-WAN, I oversee it all. Everything falls within my realm, and that's my primary focus is anything global networking at Under Armour.

Lauren M.: So you have a good sized team, but not a huge team for everything that you guys are doing. It's four people total. Right?

Chris D.: Yep. So we have four people. Well, five now. We generally stay project-based. We want to be operations-based only if something critical happens. Never hope it does. If it does, we get brought in on that side. Otherwise, we have an operations team that focuses on day-to-day operations, ticket-wise.

Lauren M.: Okay. So take me back then a couple years to 2024, when you realized the visibility was a problem. What were the drivers? What were the things that sent you looking for something like a digital twin?

Chris D.: Going back to that time, I was fresh into Under Armour. I knew we had some issues going on. Looking at device configs, you don't have time to go through every single device. It's never gonna happen. You don't have the time of day. There's not enough hours.

Anywho, we start going through it, seeing there's issues. I knew from previous experience what IP Fabric could offer. I brought IP Fabric in for a pilot to see how we can set up to uncover the 'ugly baby' to see what we had. What it ended up showing us was a lot worse than what we thought we had.

It allowed us to see the visibility of not just what we thought we had, but everything else. It didn't care if it was wireless endpoint security. It didn't care if it was SD-WAN. It didn't care what vendor you had. It looks at everything. And it calls it out.

Green is good. Blue is okay. Yellow is ‘we should really talk about this.’ Red is bad. Obviously, the goal is to shift more from red to green for each category out there.

Lauren M.: So, who cared about this? You mentioned ‘calling the baby ugly.’ That could be something that's very difficult to do, especially if you're not brand new to the team. Tell me a little bit about the the friction that you faced there and who actually cared about improving it.

Chris D.: So kind of everybody on the IT side saw it. We knew what we had. We wanted to get better. Obviously, when you have anyone looking at IT, that's generally a bad issue. You want your enterprise to work. You want the wireless to work. You want the switch to work. You want the users to be able to make phone calls, have no issues. But if leadership is looking at you, that means something's not going right. That was our driver for it.

We wanted a solution to find us a way from being reactive to all these scenarios, to being more proactive. And that is exactly what IP Fabric did for us.

Lauren M.: So there's a story that I always tell when I'm pitching. Obviously I’m a vendor. So I talk about delayed projects, and it’s something I’ve given a lot of presentations on, and a lot of theoretics. But when I talked to Chris the first time, you gave me an amazing, real-world use case, which was the EIGRP that you're pulling. So tell everyone a little bit more about what you were running into from that. What were you trying to achieve to begin with?

Chris D.: So, obviously, we're at Cisco Live. EIGRP is a Cisco routing protocol, but it's not vendor agnostic. We want to be as vendor-agnostic for routing protocols as possible anywhere out there.

So what we did was we ran IP Fabric and picked our scans off in the data centers, looking at a large project that we had in the data center. We had been peering EIGRP from core switch to firewalls—not the best idea. It's just a project that had been going on and on and on.

We knew that we had a firewall refresh coming up. That was delayed. We know that we needed to go. We wanted it to change that BGP OSPF. We want to find out exactly where we had EIGRP, what EIGRP routes we were doing in the first place. And that's where IP Fabric came in.

It pointed out all of these devices that were going to EIGRP. These are your peer neighbors inside that. So we knew once we cleared that up, that's kicked the initial leg off, so we can actually do our entire firewall migration.

Sample network topology with IP Fabric

Lauren M.: That's amazing. And, by the way, my my visual here is not of Under Armour. I'm not releasing any private data about Under Armour. This is just our internal lab, but I wanted to give everyone a visual of what a network topology in IP Fabric looks like, and the kinds of things that Chris was looking at to find the EIGRP and, of course, other things on the network.

So going a different tack here: how have you and your team actually incorporated IP Fabric into your daily lives? But then also, from a long-term perspective, what are you thinking about? What are you planning?

Chris D.: So with IP Fabric initially, the whole intention for our process was: what do we have that’s bad? How can we fix it? And what's our roadmap to get there?

We uncovered a huge solution process for us, and I harp on this all the time. I praise this part of—not a bad way, in a good way—budgeting. When every time you come through hardware refreshes, you can ask ‘What do you have? When's it gonna go End-of-Life (EoL)? When's your End-of-Support (EoS)? When do you need to refresh that in your environment? Do you know all the hardware you have in your environment?’

We kicked off our scans with IP Fabric. There's one of the scans with EoS details, End of Maintenance (EoM). All of that, it generates if you click on one of them. You can pull the entire ball of wax out of your entire environment, for everything it scans, and it'll say, ‘This is every piece of hardware you have, and when it expires. EoS is this time frame. EoL is this time. Here’s when everything should be replaced.'

Intent verification at scale with IP Fabric

Cisco does do some recommendations already inside of there saying ‘The replacement model for this is this,’ it and gives you the links to Cisco to see.

I used all of that data. It exported out with a scan based on a region—APAC, the Americas, EMEA. I can break my budgets up between those regions, and then break it up even more then that. Let me go by year. So I see 2025, '26, '27, '28.

I have a budget all the way through 2031 knowing exactly what's in our environment to be replaced. I can go to finance each year and say, ‘I have this planned. I need this to be replaced this year.’

That way, all of that budgeting is already done. That took me less than 30 minutes to do, which prior to that was days on days on days because we needed to pull the entire Cisco install base. That's hardware you haven't had, in some case, for three or four years, and you have to start sifting through that data manually. This scans every single thing in your environment. Doesn't care if it's Cisco or whoever. It'll tell you all of your data so you can stay ahead of the game on it.

Lauren M.: How else were you able to prioritize that? I mean, obviously, by the EoL timeline. But were you looking at sites? Were you looking at vendors? Were you looking at device types? What were the the types of things that you're doing to prioritize them?

Chris D.: The way that we were looking at it before was our daily monitoring tool. We would pull the hardware based on that. The other thing was we had to crawl through our Cisco install base. What did that say? And that was not clean. That was very dirty.

There's a lot of hardware you're not gonna have, but we had to keep on sifting through that all manually. There was no automated process, nothing to help us, but it wasn't just showing this switch or this router or this AP. It was showing ‘This is your switch, but also there's power supplies inside that. There's optics.’

Generally, we don't care because they're bundled in with the with the platform device. This still shows that but also filters it out, so you don't have to worry about seeing the noise.

Lauren M.: We're talking about EoL, and having to refresh hardware. I think that calls back to EIGRP; you would be in the middle of a hardware refresh project, and then you'd run into the EIGRP.

You'd stop at that hardware refresh and have to move to that and then go back, and so now multiple projects are being delayed. But tell me a little bit more about the hardware refresh and some of the things you're doing pre and post-change management.

Chris D.: We would kick everything off through IP Fabric. We do everything kind of in a bubble. We would look at things with the reverse mentality of it. If we want to do this, we want to get to here, then we would just jump straight into it just like any other organization would. We wouldn't really do the the the pre-validations of ‘What are our limitations? What do we need to get done before we get there?’

IP Fabric showed us some of that detail even as far as troubleshooting. I know one of the slides showed the the full topology check of it. So it would show your physical links that you can sit there. You can call through your device. Maybe the partner device, the other end of the device is not CDP neighbors. It's not LODP neighbors, or that's not enabled in your hardware. IP Fabric doesn't care. It'll look at that device. It'll start scrolling through and say, ‘This neighbor ships here.’

It'll look at stuff and say, ‘This is your your Layer 1 link. You're doing Layer 2 across this with VLANs, or you're doing Layer 3 here.’

But not just you're not doing Layer 3 there. You're doing Layer 3 with EIGRP, Layer 3 with OSPF, Layer 3 with iBGP, your eBGP. It would show you all of that data so you have a clean look within each one of those daily scans to help you prepare and get better for that migration.

Lauren M.: What I love about this is I left it kind of complicated. But you could turn off some of these protocols, too. So if I only wanted to see EIGRP or only OSPF, I can turn off everything else. I can turn off Layer 2 and Layer 1 and then focus on those areas only. Any other tips and tricks that you are using to zero in on what you needed?

Chris D.: That’s the main thing we would do. Like you said, we would turn off a lot of the links, clean up a lot of the noise, so you’re seeing exactly what you want to see.

If you’re doing a switch refresh, are you Layer 3 to the edge? Are you Layer 2 at the edge? Just default routes back to your core switch in that facility. Do you want to be Layer 2 on the edge? Do you want to get to a Layer 3 core? This will help you get to that because you can see where you're pushing, where you're not pushing.

We just did a massive wireless migration, for the WLCs that we're extending after the 9800s. When we did that, we made a mistake. We have EIGRP going all the way into our server farm, and we needed to flip that way to do the whole migration to convert us from EIGRP. Just basically the default routes to go through. So we don't have to worry about that. Routing stays on the core side of it.

It cleaned up the environment, simplified us, but it also it allows us to go from each site being its own site, to now, each site in a model is a replica of the other site.

Lauren M.: Well and that standardization is so important. Like you mentioned, you have five people on your team managing a global network. If you can't just remove things, then you’ve got to standardize. That's that's the other way you can simplify. So tell us a little bit more about that standardization project.

Chris D.: So the other thing is we're looking at DNS servers, NTP servers, TACAC servers, RADIUS servers. Are you configured the way that you wanna be? Do you want everything to really point to one one of the servers? Cisco's big mentality is load balance. Do you want everything to point to one ISPSN node? Probably not. You wanna load balance those nodes.

So when we kicked off IP Fabric, we worked with our great IP Fabric team to set up a custom intent check. In the Americas, we want to look at these particular nodes for DNS, and then we want these to be our backup nodes. In APAC, we want these to be our DNS nodes, and these our backup nodes. EMEA, same thing.

Then we looked at TACACS. Then we looked at NTP servers. What NTP server were we pointing at? Kick that through IP Fabric, and it generated the list.

It showed everything that was off, and it helped us from there to pull that device inventory to build out an Ansible script, and then we could push the entire Ansible script. Because IP Fabric will only pull, it won't push, which is really good for your cyber team. They're gonna like that. We built an Ansible script pulling all that data, and we'd push it out. We would update all of our equipment. I could do all three regions in the world in 15 minutes.

Lauren M.: That's pretty amazing. If someone deploys IP Fabric in their environment, what would be the first thing that you would recommend that they do to get the most value?

Chris D.: That's a big one. Coming from the leadership side, I look at budgeting. You're going to have to have a budget to do anything. IP Fabric is going to show you your budget. It's going to show you what's EoL already, what's EoS, because at that point you're not getting patches. You're at a security risk.

So you need to look at your data. What do you want to prioritize the most? You want to look at the management side of it. I think management is a big side of it.

If you've had someone pull your spanning tree data, generally, we do see spanning tree storms in the U.S. A lot of those are self-induced. If you're gonna see them, you're gonna see them.

IP Fabric will show some of that data where you're at risk, where the bridges aren't there, where you're not consistently across your organization.

Honestly, budgeting was huge.

Lauren M.: That’s huge that you were able to budget into 2031, that you were able to understand what vendor markups might be coming, and how to spend the money and prioritize that.

One more question: What would you say is dramatically different from a year ago or two years ago, before you used IP Fabric? I mean from an outcomes perspective, rather than the technical things that you were able to implement. How have you been relieved, or how has your management team been relieved?

Chris D.: So I hope this is wood, because we haven’t had a P1 in a year. We’ve had a P2—that was self-inflicted, that was my fault—but we haven’t had a P1 in over a year.

We got so much more proactive in our environment than reactive, so we're not having to chase down issues. The big thing we do is our offshore support team, which runs our daily operations, will go through IP Fabric every Thursday. We’ll export a report to our ‘Reports’ tab, generate reports for the network. They can pull that list and put everything through change control so that they following week they can be putting those changes in for remediation.

So that list that we had is now consolidating down daily and daily. Until we build more intent checks. And as we build those intent checks on top of the thousands that IP Fabric already has, we're just extending our list.  

Lauren M.: So what's the next thing that you plan to do? I mean, I heard you say, ‘Creating service requests.’ I heard about a lot of automation that's been created. What's next on your list?

Chris D.: We just did the AWS integration for the DevNet side. We're we're in a sandbox environment, so that way we can start extending to our production AWS side, and pull the data that we couldn't see there before. Because you can do end-to-end troubleshooting, you can scan it and say ‘Source, destination, what’s your flow look like the whole way through? Are you hitting a firewall rule in that statement? What's blocking your data?'

The other thing is that you can do a third-party integration, where it’ll scan your environment, see what version code you're running on those, and you can integrate those in the p-certs. What are your vulnerabilities you have in those p-certs? Right now, security is a topic everywhere we go with AI coming in. So that's the big thing that we can look at. ‘What are the p-cert vulnerabilities based on the code that we have in our environment, whether it’s our SD-WAN side, our route switch side, our nexus in the data center. All we want to see if what we’re impacted by. IP Fabric pulls it for us seamlessly.

Lauren M.: I know you told me you're busy creating custom intent checks too. You mentioned that you're actually predicting outages. Tell us a little bit about how you're doing that.

Chris D.: There's an intent check that we have. Every network engineer knows this one. You look for CRC errors. Road map, please put the CRC errors on there. So we built a custom check for the looking for CRC errors. So you can pull that data as the interface generates CRC errors. Like I said, IP Fabric doesn't care what vendor it is. It's gonna look at all that data. You would come in, click on that little box, and it'll say all 10 devices that are getting CRC errors.

This is the device. This is the interface. And you can start looking at that. It's going to be probably a bad cable, bad optic. Something is bad right there. Go ahead and get proactive. Replace that. Or maybe that was during a cutover, and the interface was flapping. It could just be something stalled. You need to clear the counters on it. If you clear the counters on the next scan, it's still incrementing You know that you can get proactive and start replacing that because the quieter leadership is towards your network, the better you are. It sounds bad, but it's a good thing.

Lauren M.: That's what we'll leave you with today. Thanks so much for being here, Chris, and for those real-world examples that you're actually using. I've been using them all week when I'm talking to people, giving them your use cases. If anyone has any questions, Chris and I will be around.

Sample end-to-end path lookup with IP Fabric

Audience Member: We’re familiar with IP Fabric, and we’ve been trying to get it into our environment. One of the things we found value in is the path analysis. Have you had to use that in any of your troubleshooting to date? Has it helped?

Chris D.: We have used it in our troubleshooting, yes. I can't give our exact topology out because it's a recorded meeting. But, yes, we have had to use it for retail stores to come back after calculating tax software. We were seeing outages. We couldn't find out what that outage was related to. We did source destination. We looked at the entire flow. We could see source to destination looking the whole way through, and we were trying to see where the drop was. Was it inside of our network, or was it outside of our network through one of our partner providers?

This allowed us to defend ourselves and say, this is not an Under Armour network issue. This is outside of our network, and what we were getting from our partner was, ‘It must be your ISP.’ So we engaged our ISP, and our ISP said, ‘No. We're clean all the way up to the next ISP.'

The partner was telling us at that point, ‘Oh, go put a ticket in with them.’ I said, ‘We don't have a contract,’ and then, ‘We can't do that. This doesn't work.' So it allowed us to really defend the network aspect of it.

Try IP Fabric yourself with our self-guided demo, or dive right in with a free trial today.