---
sourceDocument: Australia Workflow Data Fabric
sourceDocumentLink: https://www.servicenow.com/docs/r/australia/integrate-applications

 Release :

    - australia

ft:locale :

    - en-US

ft:publication_title :

    - Australia Workflow Data Fabric

ft:clusterId :

    - crint

bundleId :

    - crint

workflow :

    - Creator


---

# Amazon S3 metadata collector

# Amazon S3 metadata collector {#ariaid-title1}

Release version: Australia  
Updated August 6, 2026  
![](https://www.servicenow.com/docs/portal-asset/ico-clock) 1 minute to read  
The Amazon S3 metadata collector harvests read-only metadata from an AWS S3 account.

The collector harvests metadata from AWS S3, including buckets, objects, and their associated metadata.{#amazon-s3-metadata-collector__amazon-s3-metadata-collector-para-1}

## Authentication supported {#amazon-s3-metadata-collector__section-authentication}

The collector supports the following authentication methods. For details, see [Credentials and authentication](https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/credentials.html) in the AWS documentation.{#amazon-s3-metadata-collector__section-authentication-para-1}

* Authentication details supplied through an AWS credentials file
* Access Key ID and AWS Secret Access Key
{#amazon-s3-metadata-collector__ul_pxh_zjd_qjc}

## Metadata cataloged {#amazon-s3-metadata-collector__section-metadata-cataloged}

{#amazon-s3-metadata-collector__table-s3-metadata-cataloged__entry__2}

| Object | Information cataloged |
|-|-|
| Bucket | * ARN (Amazon Resource Name) * Region * Name * Version state * Creation date * ACL (access control list) owner ID * ACL grantee ID * ACL grant permission {#amazon-s3-metadata-collector__ul_qxh_zjd_qjc} |
| Object | * Key * ARN (Amazon Resource Name) * Region * Size * Last modified date * ACL (access control list) owner ID * ACL grantee ID * ACL grant permission * Metadata {#amazon-s3-metadata-collector__ul_rxh_zjd_qjc} |
[Table 1. Amazon S3 metadata cataloged by object type]

{#amazon-s3-metadata-collector__table-s3-metadata-cataloged}

## Relationships between objects {#amazon-s3-metadata-collector__section-relationships}

By default, the harvested metadata includes catalog pages for the following resource types. Each catalog page has a relationship to the other related resource types. If the metadata presentation for
this data source has been customized, you might see other resource pages and relationships.{#amazon-s3-metadata-collector__section-relationships-para-1}
{#amazon-s3-metadata-collector__table-s3-relationships__entry__2}

| Resource page | Relationship |
|-|-|
| S3 Bucket | S3 Object |
| S3 Object | S3 Bucket |
[Table 2. Amazon S3 resource page relationships]

{#amazon-s3-metadata-collector__table-s3-relationships}

## Limits for S3 buckets {#amazon-s3-metadata-collector__section-limits}

The collector has a default limit of 10,000 objects per bucket. If a bucket exceeds this limit, the collector skips the bucket, harvests no metadata for it, and logs a warning message.{#amazon-s3-metadata-collector__section-limits-para-1}

To increase the limit beyond the default, set the --max-resources parameter in your collector command. The maximum value for this parameter is 10,000,000 (ten million). If the total
contents across all buckets and objects exceed this limit, the collector stops cataloging additional buckets and logs a warning message.{#amazon-s3-metadata-collector__section-limits-para-2}
* **[Prepare to run the Amazon S3 collector](https://www.servicenow.com/docs/gOJCnvFqqDlorRt_ftT3yg)**   
  Create an AWS IAM user with the permissions required for the metadata collector to harvest metadata from Amazon S3 buckets and objects.
* **[Create an Amazon S3 metadata collector](https://www.servicenow.com/docs/kK0cmMs3JBxCBMXjLfMLaw)**   
  Use a metadata collector to import Amazon S3 bucket and object metadata into the data catalog.