Handling import failures for large GCP resources

  • 릴리스 버전: Australia
  • 업데이트 날짜 2026년 03월 12일
  • 소요 시간: 6분

    • Manage the data source failure when importing resources from GCP.

    When importing large GCP resources, the data sources in the Service Graph Connector for GCP might fail because of a Direct REST response exceeding the 1024-MB limit. For more information, see REST step.

    You can use one or more of the following options to handle the REST response size limitation error in the Service Graph Connector for GCP:

    Save the response as an attachment

    With the save response as an attachment feature, the response of each data source is temporarily stored as an attachment and after the response is read, the attachment is deleted automatically. The save response as an attachment feature is activated by default and is triggered by the sn_gcp_integ.save_rest_response_as_attachment system property.

    As a user with the admin role, you can deactivate the save response as an attachment feature by setting the sn_gcp_integ.save_rest_response_as_attachment property to false. When the feature is deactivated and the response size is within the limit, the data sources run normally.

    Modify the batch size

    By default, the maximum batch size of the environment variable for a data source is set to 100 records. You can reduce or increase the batch size based on the data volume. To manage the batch size of a data source, you must add a system property of integer type for the data source as described in the following table.

    표 1. System properties for handling the batch size of Service Graph Connector for GCP data sources
    Data source System property
    SG-GCP Cloud Database sg_gcp_cloud_db_batch_size
    SG-GCP Cloud Function sg_gcp_cloud_function_batch_size
    SG-GCP Cloud Object Storage sg_gcp_object_storage_batch_size
    SG-GCP Kubernetes Cluster sg_gcp_kubernetes_cluster_batch_size
    SG-GCP Kubernetes Cluster Role sn_gcp_integ.sg_gcp_kubernetes_cluster_role_batch_size
    SG-GCP Kubernetes Cluster Role Binding sn_gcp_integ.sg_gcp_kubernetes_cluster_role_binding_batch_size
    SG-GCP Kubernetes Deployment sg_gcp_kubernetes_deployment_batch_size
    SG-GCP Kubernetes Namespace sg_gcp_kubernetes_namespace_batch_size
    SG-GCP Kubernetes Node sg_gcp_kubernetes_node_batch_size
    SG-GCP Kubernetes Pod sg_gcp_kubernetes_pod_batch_size
    SG-GCP Kubernetes Replicaset sg_gcp_kubernetes_replicaset_batch_size
    SG-GCP Kubernetes Service sg_gcp_kubernetes_service_batch_size
    SG-GCP Load Balancer sg_gcp_load_balancer_batch_size
    SG-GCP Load Balancer Health Service sg_gcp_lb_health_service_batch_size
    SG-GCP Load Balancer Pool sg_gcp_lb_pool_batch_size
    SG-GCP Load Balancer Pool Member sg_gcp_lb_pool_member_batch_size
    SG-GCP Load Balancer Service sn_gcp_integ.sg_gcp_lb_service_batch_size
    SG-GCP Network sg_gcp_network_batch_size
    SG-GCP Kubernetes Node Pool sn_gcp_integ.sg_gcp_kubernetes_node_pool_batch_size
    SG-GCP Security Group sg_gcp_security_group_batch_size
    SG-GCP Software Inventory sg_gcp_software_inventory_batch_size
    SG-GCP Storage Volume sg_gcp_storage_volume_batch_size
    SG-GCP Storage Volume Snapshot sg_gcp_storage_vol_snapshot_batch_size
    SG-GCP Subnet sg_gcp_subnet_batch_size
    SG-GCP VM Instance sg_gcp_vm_instance_batch_size
    주:
    You must add these system properties as they aren't available by default. You must have the admin role to add or modify a system property. For more information, see Add a system property.