Data collection and discovery using Netflow

  • Release version: Australia
  • Updated March 12, 2026
  • 2 minutes to read
    • Summarize
    Summarized using AI
    This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete.

    Summary of Data Collection and Discovery Using Netflow

    Service Mapping leverages the Netflow protocol to enhance data collection about Configuration Items (CIs) and their connections, complementing traditional methods like netstat and lsof commands. This traffic-based discovery method provides comprehensive insights into network interactions, aiding in the management and mapping of IT resources.

    Show full answer Show less

    Key Features

    • Traffic-Based Discovery: Utilizes TCP-related data collected via netstat, ss, and lsof commands, alongside Netflow and VPC Flow Logs for enriched information.
    • Data Collection Setups: Offers two configurations:
      • Testing Purposes: Involves manual data import from a Netflow Collector placed on a separate server from the MID Server.
      • Standard Operation: Enables fully automated data collection with the Netflow Collector on the same server as the MID Server.
    • Data Processing Flow: The Netflow daemon collects data from network switches, which is then written into an nfdump output file for processing by the MID Server.

    Key Outcomes

    By configuring Service Mapping with the Netflow protocol, customers can expect:

    • Automated and manual data collection options tailored to their operational needs.
    • Enriched CI connection information through additional data from netstat and saflowconnection tables.
    • Enhanced visibility and mapping of network interactions, supporting better resource management and operational efficiency.

    Service Mapping can perform discovery based on data collected using the Netflow protocol. Netflow is a protocol that Service Mapping can use to collect data about CIs and their connections along with Netstat and lsof commands.

    Using the Netflow protocol for collecting data is one of the traffic-based discovery methods. Other methods deployed by Service Mapping are using netstat and lsof commands and the VPC Flow Logs. For more information, refer to Traffic-based discovery in Service Mapping.

    In base systems, which are the default or standard configurations, traffic-based discovery relies solely on TCP-related data collected using the netstat, ss, and lsof commands. Discovery based on Netflow and VPC logs requires additional configuration. You can enrich your traffic-based discovery by configuring Service Mapping to use the Netflow protocol.

    The component, which receives data in the Netflow format is the Netflow Collector. Its location depends on whether you configure data collection for testing purposes or standard operation:
    For the test purposes
    This setup results in half automated data collection flow, where Service Mapping imports data only if you manually copy it from the Netflow Collector. You place the Netflow Collector on a server inside your organization network. This must be a server different from the server hosting the MID Server. You configure and test this setup as described in Configure onetime data import using Netflow for testing purposes.
    For standard operation
    This setup results in fully automated data collection flow, where all involved components send, collect and analyze data automatically. You place the Netflow Collector on the same server as the MID Server inside your organization network. For instructions, see Configure data collection using Netflow.
    Netflow-based discovery has the following flow:
    1. The Netflow daemon runs and receives data from switches communicating with servers in the organization. The Netflow Collector writes received data from the Netflow daemon.
    2. The server, hosting the Netflow collector, uses the Netflow nfdump utility to write the data into the nfdump output file. This file summarizes the raw data on all switches used for server communication.
      Figure 1. Collecting data and writing it into the nfdump output file

      Standard Netflow configuration: collecting data and writing it into the nfdump output file
    3. In testing setups, where the Netflow Collector is located not on the same server as the MID Server, you may need to convert the nfdump into the gzip format. Then you must manually copy the raw data in the nfdump output file onto the MID Server.
      Figure 2. Copying the nfdump output file onto the MID Server

      Standard Netflow configuration: copying the nfdump output file onto the MID Server
    4. The MID Server processes the raw data in the nfdump output file and places the processed information onto the ECC queue.
      Figure 3. Analyzing the raw data and placing it at the ECC Queue

      Standard Netflow configuration: analyzing the raw data and placing it at the ECC Queue
    5. A sensor retrieves the processes data from the ECC queue and writes it into the Flow Connection [sa_flow_connection] table.

    6. Whenever Service Mapping checks the ECC queue and receives information on a discovered CI, it checks these tables for any data on outbound connections related to the CI: the cmdb_tcp and sa_flow_connection tables. If these two tables contain unique data that patterns did not discover, Service Mapping enriches the information about the CI connections and adds them to the map.

      Figure 4. Service Mapping retrieves data from the sa_flow_connection table

      Standard configuration: Service Mapping retrieves data from the sa_flow_connection table