Introducing the IP Fabric NetBox Plugin

Background:

Before getting into the details, there are a few necessary background details that need to be understood.

What is NetBox used for?

NetBox is the leading solution for modeling and documenting modern networks. By combining the traditional disciplines of IP address management (IPAM) and datacenter infrastructure management (DCIM) with powerful APIs and extensions, NetBox provides the ideal "source of truth" to power network automation. Read on to discover why thousands of organizations worldwide put NetBox at the heart of their infrastructure.

See: https://docs.netbox.dev/en/stable/

What is a Plugin?

Secondly, what is a NetBox plugin?

Plugins are packaged Django apps that can be installed alongside NetBox to provide custom functionality not present in the core application. Plugins can introduce their own models and views, but cannot interfere with existing components. A NetBox user may opt to install plugins provided by the community or build his or her own.

See: https://docs.netbox.dev/en/stable/plugins/

Why is it important?

Network documentation and the use of tools like IP Fabric and NetBox are essential components of successful automation initiatives. Properly documenting the network provides invaluable visibility, allowing network administrators to understand device configurations and connections, troubleshoot issues, plan upgrades, and enhance security. Moreover, tools like IP Fabric and NetBox offer structured data that automation scripts rely on, enabling tasks such as configuration management, provisioning, and optimization. This combination of documentation and automation not only reduces manual errors but also accelerates processes, ultimately leading to more efficient and reliable network management.

Many of our valued customers have embraced NetBox as a tool for holding source-of-truth information on their journey towards network automation. What sets IP Fabric apart is its ability to gather an extensive array of network data, making it a perfect companion to NetBox. There are two primary scenarios where synchronizing data from IP Fabric into NetBox proves invaluable. The first scenario is when customers are starting fresh with a NetBox deployment. They're new to the tool and want to enrich it with comprehensive information from their network, which IP Fabric readily provides. The second scenario arises when customers already have an established NetBox instance. In this case, they seek to validate its accuracy against the real network through IP Fabric.

Introducing the IP Fabric NetBox Plugin

In line with our commitment to enhancing our user experience, we're thrilled to introduce the IP Fabric NetBox Plugin. This innovative plugin is designed to streamline and simplify the process of synchronizing vital network information from IP Fabric directly into NetBox, ensuring that your NetBox instance remains up-to-date and reflective of your actual network infrastructure.

We've made the plugin documentation readily accessible through our public documentation, please see this for installation instructions. With this plugin, users can seamlessly synchronize the following NetBox models:

The data synchronization process

The NetBox Plugin allows for the configuration of multiple instances of IP Fabric as sources within NetBox. This functionality enables our customers with multiple IP Fabric instances to seamlessly synchronize data, regardless of its source.

Sources Detail Page

Dividing the process of ingesting snapshot information and network data into two distinct components empowers users to synchronize details about the snapshot IP Fabric, all without the need for real-time connectivity to the IP Fabric API. This approach allows for the seamless presentation of information within NetBox.

SCR 20230922 khsp

After synchronizing snapshots, an ingestion job can be configured.

Editing IP Fabric Sync Netbox Demo

This defines the network information our customers wish to synchronize into NetBox. They can specify the snapshot from which to ingest data, as well as the sites to include. Additionally, customers have the flexibility to choose the models they want to import into NetBox. When selected, the plugin can either update existing models or create new ones in NetBox. When deselected, the plugin will only perform a 'get' operation for the model. Furthermore, customers can schedule these ingestions for future dates or set them to repeat at specific intervals. Any tag associated with an ingestion job will be added to the new models.

SCR 20230922 krha

Creating an ingestion job does not imply automatic data synchronization. Instead, the job must be initiated either through a predefined schedule or manually via the 'Sync' button on the ingestion page.

It's important to emphasize that any data ingestion into NetBox does not automatically overwrite existing data. We utilize a feature within NetBox that enables the plugin to propose changes in a branch, akin to how GIT operates. For further details, please refer to the official NetBox documentation.

After initiating a synchronization job, a branch is created where the progress and details of the process can be observed.

2023 09 22 11 05 04 482176 00 00 4

The 'Changes' tab provides information about the modifications that would be applied to the NetBox database if this branch were to be merged.

SCR 20230922 kygt

Pretty cool right? I think so. It's important to highlight a key part of the process that makes this work and that is transform maps.

Transform Maps

What are transform maps? Transform maps are like a carefully crafted recipe. This recipe serves as a guide, orchestrating the process of extracting data from IP Fabric and delicately depositing it into the appropriate “containers” represented by Django models within NetBox. Its primary objective is to guarantee precise and efficient organization of the data, meticulously tailored to meet your network management requirements. In essence, it acts as a bridge, harmonizing disparate data formats and structures, ensuring a seamless integration of your network data with your NetBox database.

The concept of transform maps is not new; it has been widely employed in various tools for transforming data from one format to another, a practice commonly referred to as ETL (Extract, Transform, Load). This concept draws substantial inspiration from ServiceNow and their implementation of Transform Maps.

Why did we choose to use transform maps?

During our plugin testing phase with customers, the feedback was overwhelmingly positive. However, a few recurring questions and comments arose, such as, "Can we input data into this specific NetBox field?" or "We don't use NetBox in that manner." These inquiries were primarily linked to the plugin's fixed field structure for various data elements.

The introduction of transform maps proved to be a game-changer. They provided a solution that eliminated the need for custom development for each customer. With transform maps, customers gained the ability to modify our default transform maps to manipulate and designate different data destinations as per their requirements. This newfound flexibility has proven to be a significant advantage, catering to a wider range of customer needs and preferences.

Example with structured data:

The data below is an example of the data used to create a site in Netbox.

{
        "siteName": "Site 1",
        "devicesCount": 173,
        "usersCount": 7075,
        "stpDCount": 10,
        "switchesCount": 165,
        "vlanCount": 113,
        "rDCount": 2,
        "routersCount": 12,
        "networksCount": 77
}

The transform map assumes a crucial role in aligning the IP Fabric site table with NetBox's Site model. This synchronization is achieved through the association of transform fields with the transform map. These fields serve as vital connectors, establishing mappings between keys in the dictionary and specific fields within NetBox's database model.

Each transform field possesses two fundamental attributes: a source field (corresponding to a key in the IP Fabric dictionary) and a target field (representing a NetBox model field). These fields sync harmoniously, making it easy to transfer and map data during the ingestion process.

For situations requiring more intricate transformations, the system offers enhanced flexibility. Users can opt to supply a Jinja template, thereby enabling additional functionalities such as slugifying the name of a site or executing supplementary model lookups. This capability empowers the tailoring of the data transformation process to address specific, advanced requirements as necessary.

The following template would result in the siteName being site-1.

{{ object.siteName | slugify }}

For more of a detailed explanation on how transform maps work please see our documentation.

What Now?

If you're an IP Fabric customer using NetBox and haven't been contacted by your solution architect, we encourage you to reach out and express your interest in exploring the NetBox plugin. Should you encounter any challenges or have questions, please don't hesitate to communicate them either directly to your solution architect team or through Gitlab Issues. Your feedback and engagement are valued, and we're here to assist you in making the most of IP Fabric and the NetBox plugin.

Update (October 2023): Take a look at our freshly built NetBox plugin instead!

What is NetBox

NetBox is an infrastructure resource modeling (IRM) application designed to empower network automation. Initially conceived by the network engineering team at DigitalOcean, NetBox was developed specifically to address the needs of network and infrastructure engineers.

https://docs.netbox.dev/en/stable/

Notably, NetBox is an open-source tool. Everyone has free access to the code and many simple deployment options are offered. For my testing purposes, I decided to use its docker image. It took me about 5 minutes to deploy and get the NetBox ready.

What is IP Fabric

The IP Fabric is the Automated Network Assurance Platform that helps enterprises empower their network and security teams to discover, model, verify and visualize large-scale networks within minutes. Its main goal for any network infrastructure is to regularly capture the current network state. It provides another layer of abstraction for its users to access the network state data and it is vendor-agnostic!

In plain words, the IP Fabric will provide the model for you! And one of its biggest advantages is its standardized and well-documented API. Imagine capturing inventory, routing tables, or security policies from all discovered firewalls in a single request. That's how useful its API can be.

Understanding IP Fabric's API is fairly simple. Every provided data set (inventory, routing information, multicast data, security policies, part numbers, management protocols, and many many more) has its own dynamic documentation that will provide all necessary to build your API request including full payload information, description of all its properties and more. All you need to start your automation journey.

ipf nb apidoku
API documentation inside IP Fabric's device inventory

The importance of an accurate data model

Let's divert the flow a bit and think about the importance of a reliable data model. When we talk about the data model, we mean the dataset that represents the network state.

The need for an accurate data model increases with complexity. Let's consider managing 100 network devices, with a couple of minor changes per month. We keep all the states (routing, switching, policies, ..) in our head, and from time to time we update the spreadsheet and our Visio documentation. Great!

Then what about 1000 network devices with hundreds of minor changes a month managed by a larger team? The previous concept doesn't scale and when an engineer leaves, the knowledge follows. That's why the standardized data model (or it may be called the Source of Truth) should be part of every network team, including operations, development, or architecture.

Data model management

Now we agree that having an accurate network model is a must to navigate complexity. Then we have two main options to manage it.

1 - Standardize and hastily follow processes around any network change and make sure that everyone updates the CMDB after every change. It can work and it may even scale, but we are still assigning routine work to humans - and we are not good at it!

2 - Automate data management. We know that NetBox is an infrastructure resource modeling (IRM) application, but it doesn't have the discovery mechanism on its own. We still need to provide the data. In the following part, we will use IP Fabric's discovery mechanism and its API to read the network data and move them over to the NetBox with a sample script.

Groupe de masques 26

Get IP Fabric

Try our self-guided demo and discover how to increase 
your networks visibility & get better time efficiency.
Free Demo | Zero Obligation
Try our Demo

Synchronizing IP Fabric's data into NetBox

Part 1 - Discovery of the network with IP Fabric

IP Fabric’s lightning-quick intelligent network discovery process empowers you with deep insight into the workings of your network. Baseline every device and path, configuration, and security policy automatically, equipping you and your team with the knowledge and insight to support, maintain, and develop the most complex of networks.

https://ipfabric.io/solution/network-visibility-and-assurance/

As we mentioned before, with IP Fabric you can fully automate the discovery process and have all essential network state data on a silver plate. There are about 2000+ parameters for a single device we can get including the inventory data, IP addresses, VRFs, VLANs, interfaces (standardized/original), routing data, policies, and many more. IP Fabric maintains regular network state updates on its own.

Part 2 - Transform the data

So if there's one API endpoint in IP Fabric to get the device inventory data and a second API endpoint in NetBox to provide data for its inventory. Then it's just two simple requests and the job is done, correct? Technically yes, practically it's not that simple! We need to add a transformation logic.

The schema below depicts the Network infrastructure on the left. IP Fabric captures the data from the existing network over SSH or API (AWS, Azure, NSX, ..) and provides the data over API. Then:

ipf python nbWEBP 1
Synchronize your IP Fabric data with NetBox 1

Reading IP Fabric's data is simple, but to create inventory in NetBox, we need to think about the data model differences first. And second, we need to create a proper mapping structure in our code. For example:

IP Fabric's properties should be mapped correctly to create desirable results in the NetBox. Just to mention a few issues we can come into during our implementation phase:

And there can be more, which is the main reason to spend time on planning before execution.

Part 3 - Sample code to create new inventory in NetBox

For everyone who is looking for an easy way to start, I have created a sample code to update NetBox with IP Fabric's inventory data. You can find and freely use the repository on GitHub. It consists of a handful of functions to add/remove standard inventory data or to populate the NetBox inventory completely. The prerequisites are:

ipfabric-netbox sample code on GitHub
ipfabric-netbox sample code on GitHub

Good luck and enjoy!

If you have found this article helpful, please follow our company’s LinkedIn or Blog, where more content will be emerging. If you would like to test our solution to see for yourself how IP Fabric can help you manage your network more effectively, please schedule a demo with our team: Request a Demo.

We're Hiring!
Join the Team and be part of the Future of Network Automation
Available Positions
IP Fabric, Inc.
115 BROADWAY, 5th Floor
NEW YORK NY, 10006
United States
This is a block of text. Double-click this text to edit it.
Phone : +1 617-821-3639
IP Fabric s.r.o.
Kateřinská 466/40
Praha 2 - Nové Město, 120 00
Czech Republic
This is a block of text. Double-click this text to edit it.
Phone : +420 720 022 997
IP Fabric UK Limited
Gateley Legal, 1 Paternoster Square, London,
England EC4M 7DX
This is a block of text. Double-click this text to edit it.
Phone : +420 720 022 997
IP Fabric, Inc. © 2024 All Rights Reserved