Data transformation for the Microsoft Threat and Vulnerability Management Vulnerability Integration

  • Release version: Washingtondc
  • Updated February 1, 2024
  • 8 minutes to read

    • After you identify the data that you want to import, the data is retrieved from the ServiceNow® Microsoft Threat and Vulnerability Management (MS TVM) application, processed through a set of data sources, and transformed in your instance.

    During installation, normalized severity maps are installed in the Normalized Severity Mapping module. These maps transform imported Microsoft third-party vulnerability severity levels to standard severity levels for processing in your instance. For information about creating severity maps, see Create a Vulnerability Response severity map.

    MS TVM Machines Import

    The data from the imported machines is first loaded into the MS TVM Machines Import [sn_vul_msft_tvm_machines_import] table.

    The MS TVM Machines Transform is used to transform the imported machines information.
    Note:
    Changes to this transform map alter how data from the MS TVM Machines import is processed.
    To access this transform map, navigate to System Import Sets > Transform Maps and search for Microsoft TVM Machines Transform.

    The following table lists the transform map fields by integration.

    Table 1. MS TVM Machines transform map fields
    Source field Target field Description
    u_id source_id Unique ID for assets. This ID maps to the source_id of the discovered item record.
    u_ipaddresses.macAddress mac_address MAC Address that is mapped from the API to the host mac address field of the cmdb_ci record.
    u_ipaddresses.ipAddress ip_address IP Address field that is mapped to the IP address field of the cmdb_ci record.
    u_lastseen last_scan_date Field that maps to the last_scan_date field on the discovered item record.
    u_machinetags Tags that are saved in sn_sec_cmn_host_tag. The mapping from tags to assets is saved in sn_sec_cmn_m2m_src_ci_tag.
    u_osplatform os Field that maps to the os field on the cmdb_ci record.
    u_computerdnsname fqdn Field that maps the dnsname field from the API to the fqdn field on the cmdb_ci record.

    The following transform scripts are run during the transformation process.

    MS TVM Machines transform map script timing and purpose

    When the script is run Purpose
    onStart (when an import set has started transformation). Script that is used to initialize the values in the scope variable (sn_vul_msft_tvm) for the integration process. This script is for internal use and should not be modified or deleted.
    onBefore (before an import set has completed transformation). Script that is used to update values in the host and verify whether the host exists. Based on the results, this script modifies the values in the scope variable (sn_vul_msft_tvm). This script is for internal use and should not be modified or deleted.
    onComplete (when an import set has completed transformation). Script that is used to set the number of CIs created, updated, and ignored. This script is for internal use and should not be modified or deleted.

    The MicrosoftTVMMachinesProcessor script include is called from the onBefore transform script. It takes the output from the Microsoft TVM machines' integration and transforms it into a CI. Any changes to this script include may alter the transformation of the Microsoft TVM machines' data in the CI and Discovered item table.

    MS TVM Vulnerabilities integration

    The imported vulnerabilities data is first loaded into the Microsoft TVM CVE (Vulnerabilities) Import [sn_vul_msft_tvm_vulnerabilities_import] table.
    Note:
    Changes to this transform map alter how data from the MS TVM Vulnerabilities import is processed.
    To access this transform map, navigate to System Import Sets > Transform Maps and search for the Microsoft TVM Vulnerabilities Transform.

    The following table lists the transform map fields by integration.

    Table 2. Microsoft TVM Vulnerabilities transform map fields
    Source field Target field Description
    u_id id Maps to the ID column of the sn_vul_entry record.
    u_severity source_severity Maps the Severity field to the severity. The default value is 5.
    u_publishedon date_published Maps the u_publishedon field to the published date.
    u_publicexploit public_exploit Maps the u_publicexploit provided by the scanner to the public exploit column in the vulnerability entry table.
    u_cvssv3 v3_base_score Maps the cvssv3 score to the v3 base score in the vulnerability entry record.
    u_description summary Maps the description to the summary field in the vulnerability entry record.
    u_exploitinkit malware_kit Maps the u_exploitinkit field to the malware kit in the Exploit table.
    u_exploittypes type Maps the exploit type to the type in the Exploit table.
    u_exploitverified is_exploit_verified Maps the u_exploitverified field to the verified exploit in the Exploit table.
    u_exploituris exploit_links Maps the u_exploituris field to the exploit links in the Exploit table.

    The following transform scripts are run during the transformation process.

    When the script is run Purpose
    onStart (when an import set has started transformation). Script that is used to initialize the values in the scope variable (sn_vul_msft_tvm) for the integration process. This script is for internal use and should not be modified or deleted.
    onBefore (before an import set has completed transformation). Script that is used to create or update the values in the NVD or the third-party entry table. This script is for internal use and should not be modified or deleted.
    onComplete (when an import set has completed transformation). Script that is used to set the values of the new items that were created and the items that have been updated and ignored. This script is for internal use and should not be modified or deleted.

    MS TVM Recommendations import

    The imported recommendation data is first loaded into the MS TVM Recommendations Import [sn_vul_msft_tvm_recom_import] table.
    Note:
    Changes to this transform map alter how the data from the MS TVM Recommendations import is processed.
    To access this transform map, navigate to System Import Sets > Transform Maps and search for MS TVM Recommendation Transform.

    The following table lists the transform map fields by integration.

    Table 3. MS TVM Recommendation transform map fields
    Source field Target field Description
    u_recommendedvendor recommended_vendor Maps the u_recommendedvendor field to the Vendor column.
    u_weaknesses weaknesses Maps the u_weaknesses field to the Weaknesses column.
    u_exposedmachinescount src_exposed_machines_cnt Maps the u_exposedmachinescount field to the Exposed machines count column.
    u_status status Maps the status to the Status field in the recommendation record.
    u_productname product_name Maps the u_productname field to the Product name in the recommendation record.
    u_nonproductiv_impactedassets non_prod_impacted_assets Maps the u_nonproductiv_impactedassets field to the Impacted assets column in the recommendation record.
    u_activealert active_alert Maps the u_activealert field to the Active alert column in the recommendation record.
    u_recommendedversion recommended_version Maps the u_recommendedversion field to the Recommended version column in the recommendation record.
    u_totalmachinecount total_machine_count Maps the u_totalmachinecount field to the Total machine count column in the recommendation record.
    u_exposureimpact exposure_impact Maps the u_exposureimpact field to the Exposure impact column in the recommendation record.
    u_recommendationname recommendation_name Maps the u_recommendationname field to the Recommendation name column in the recommendation record.
    u_subcategory subcategory Maps the u_subcategory field to the Subcategory column in the recommendation record.
    u_id source_id Maps the recommendation ID from MS TVM to the Source ID column.
    u_remediationtype remediation_type Maps the u_remediationtype field to the Remediation type column in the recommendation record.
    u_relatedcomponent related_component Maps the u_relatedcomponent field to the Related component column in the recommendation record.
    u_recommendedprogram recommended_program Maps the u_recommendedprogram field to the Recommended program column in the recommendation record.
    u_recommendationcategory recommendation_category Maps the u_recommendationcategory field to the Recommendation category column in the recommendation record.
    u_publicexploit public_exploit Maps the u_publicexploit field to the Public exploit column in the recommendation record.
    u_vendor vendor Maps the u_vendor field to the Vendor column in the recommendation record.
    [Script] integration_instance Name of the instance from which the recommendation is imported.
    [Script] sys_domain Domain in which this record is imported.

    The following transform scripts are run during the transformation process.

    Table 4. MS TVM Recommendation transform map script
    When the script is run Purpose
    onStart (when an import set has started transformation). Script that is used to initialize the values in the scope variable (sn_vul_msft_tvm) for the integration process. This script is for internal use and should not be modified or deleted.
    onBefore (before an import set has completed transformation). Script that is used to update values in the recommendations and verify whether the recommendations exist. This script is for internal use and should not be modified or deleted.
    onComplete (when an import set has completed transformation). Script that is used to set the values of items created, updated, and ignored. This script is for internal use and should not be modified or deleted.

    MS TVM Machine Vulnerabilities import

    The MS TVM Machine Vulnerabilities transform map is used to transform open and fixed vulnerabilities information that is imported from MS TVM.
    Note:
    Changes to this transform map alter how data from the MS TVM Machine Vulnerabilities Import is processed.
    To access the MS TVM Open and Fixed Vulnerabilities transform maps, navigate to System Import Sets > Transform Maps and search for the MS TVM Machine Vulnerabilities Transform.

    The following table lists the transform map fields by integration.

    Table 5. MS TVM Machine Vulnerabilities transform map fields
    Source field Target field Description
    u_id detection_key Maps the u_id field to the Detection key column in the detection table.
    u_diskpaths proof Maps the u_diskpaths field to the Proof column in the detection table.
    u_registrypaths proof Maps the u_registrypaths field to the Proof column in the detection table.
    u_recommendedsecurityupdateid preferred_solution Maps the u_recommendedsecurityupdateid field to the Preferred solution column in the vulnerable item table, if the solution exists with the same ID in the sn_vul_solution table.
    u_recommendationreference recommendation Maps the u_recommendationreference field to the Recommendation column in the vulnerable item table.
    u_cveid vulnerability Maps the u_cveid field to the Vulnerability column in the vulnerable item table.
    u_status source_status Maps the u_status field to the Source status column in the detection table.
    u_eventtimestamp temporal_score Maps the u_eventtimestamp field to the Last found column in the vulnerable item table.
    u_lastseentimestamp last_seen Maps the u_lastseentimestamp field to the Last seen column in the vulnerable item table.
    u_firstseentimestamp first_seen Maps the u_firstseentimestamp field to the First seen column in the vulnerable item table.
    u_recommendedsecurityupdate solution_summary Maps the u_recommendedsecurityupdate field to the Solution summary column in the vulnerable item table.
    u_recommendedsecurityupdateurl solution_summary Maps the u_recommendedsecurityupdateurl field to the Solution summary column in the vulnerable item table.

    The following transform scripts are run during the transformation process.

    When the script is run Purpose
    onStart (when an import set has started transformation). Script that is used to initialize the values in the scope variable (sn_vul_msft_tvm) for the integration process. This script is for internal use and should not be modified or deleted.
    onBefore (before an import set has completed transformation). Script that is used to check if the Vulnerability Entry and Detections exist. If not, these records are created in their respective tables. This script is for internal use and should not be modified or deleted.
    onComplete (when an import set has completed transformation). Script that is used to update the count of VIs and Detections as imported from MS TVM. This script is for internal use and should not be modified or deleted.