Create a Snowflake metadata collector

Create a collector to import metadata from Snowflake.

Before you begin

Before you begin, verify the following:

• A MID Server is setup for the collectors. For more information, see MID Server for metadata collectors.
• All per-requisite tasks are completed. For more information, see Prepare to run the Snowflake collector.
• Role required: connection-admin

Procedure

1. Navigate to All > Workflow Data Fabric > Workflow Data Fabric Home.
2. Select the Connect Hub icon in the left sidebar.
3. Select Create > Metadata collector.
4. From the System list, select Snowflake.
5. From the Connection type list, select one of the following:
   1. Select New connection to configure a new connection.
   2. Select Existing connection to reuse an existing connection and select an existing connection from the Connections list.
      The configuration form is filled with details from the existing connection. The name is appended with the word Copy and sensitive details like password aren't copied.
6. Complete the form.

   Table 1. Snowflake metadata collector form

   Field	Description
   Connection name	Unique identifier for the connection. This field can't be modified once the connection is established.
   Short description	Purpose and details of the connection.

7. Configure the authentication options.

   Table 2. Authentication options

   Field	Description
   Authenticate using username and password
   Username	Username to use to make the JDBC connection.
   Password	Password of the user.
   Authenticate using a private key
   Username	Username to use to make the JDBC connection.
   Snowflake private key file	Upload the private key file to use for authentication.
   Private key file password	Password for the private key file, if the key is encrypted and a password is set.

8. Select from Collect all schemas and Specify which schema to collect.

   Table 3. Schema collection options

   Field	Description
   Collect all schemas
   Collect all schemas	Catalog all schemas to which the user has access.
   Exclude Schema	Name or regular expression of the database schema to be excluded.
   Include Information Schema	Include the database's Information Schema in catalog collection.
   Specify which schema to collect
   Specify which schema to collect	Catalog only the specified schemas.
   Schema	Name of the database schema to catalog.

9. Enter the Snowflake connection details.

   Table 4. Connection details

   Field	Description
   Server	Hostname of the database server to connect to.
   Server port	Port of the database server (if not the default).
   Database	Name of the database to connect to. Specify multiple databases by adding one value per line.
   Snowflake Role	The role used to execute the query.
   Excluded database	Name or regular expression indicating databases not to catalog when the Database field is empty. Note: This parameter is ignored if the Database field is specified.

10. Configure the statistics and sampling options.

    Table 5. Harvesting scope and lineage options

    Field	Description
    Collect Data Metric Function Information	Option to harvest metadata about data metric functions and their associations to tables.
    Data Metric Observation Count	Number of most recent data metric observations to harvest. Ignored unless Collect Data Metric Function Information is enabled. Default: 1
    Enable column statistics collection	Option to harvest column statistics, that is, data profiling. Note: Activating this option may extend the collector running time because the collector reads table data to gather profiling metadata.
    Target sample size for column statistics	Number of rows sampled for computing column statistics and string-value histograms. For example, to sample 1000 rows, set the value to 1000. Default: 100000 Note: Applies only if Enable column statistics collection is enabled.
    Collect Snowflake Table Usage Information	Option to harvest metadata about Snowflake table usage in queries, including popularity. For each table in the database, calculates the percentage of tables queried no fewer times than the subject table.
    Table Usage Lookback Days	Number of days in the past from which to begin harvesting table usage. Default: 7 days
    Disable Lineage collection	Option to skip harvesting intra-database lineage metadata.
    Disable Extended Metadata collection	Option to skip harvesting extended metadata for data asset types such as database, schema, table, columns, functions, stored procedures, user-defined types, and synonyms. Basic metadata for these data asset types is still harvested.

11. Configure the harvesting scope and limits options.

    Table 6. Harvesting scope and limits options

    Field	Description
    Collect Snowflake Tag Information	Option to harvest metadata about Snowflake tags in or associated with the database specified in the database setting.
    Collect All Snowflake Tag Information	Option to harvest metadata about Snowflake tags regardless of the database in which they reside.
    Collect Snowflake Policy Information	Option to harvest metadata about Snowflake masking and row-access policies in or associated with the database specified in the database setting.
    Collect All Snowflake Policy Information	Option to harvest metadata about Snowflake masking and row-access policies regardless of the database in which they reside.
    Collect Streamlit App Information	Option to harvest metadata about Snowflake Streamlit applications.
    Enable Sample String Values collection	Option to harvest sample values and histograms for columns containing string data. Note: Applies only if Enable column statistics collection is enabled.
    Exclude system functions	Option to exclude built-in system functions from harvesting.

12. Configure the connection and reliability options.

    Table 7. Connection and reliability options

    Field	Description
    Snowflake Warehouse	Snowflake warehouse to use when connecting. Default: User's assigned default warehouse
    Server environment	Friendly name for the environment in which your database server runs. Useful when the server name is localhost and you need to differentiate it from other environments.
    Database ID	Unique identifier for this database, used to generate the database ID when the database name is not sufficiently unique.
    JDBC properties	JDBC driver properties passed to the driver connection.
    SQL parsing timeout	Timeout in seconds for SQL parsing during lineage collection. Default: 60

13. Select Save.

Result

The metadata collector is created and appears on the Connectors page with a Configured status. It is now ready to connect to the source system and harvest metadata.

What to do next

After creating the collector, you can perform any of the following tasks:

• Run the collector manually to harvest metadata immediately. See Run metadata collectors manually.
• Automate metadata collection by scheduling regular collector runs. See Schedule metadata collector runs.
• Monitor execution status and troubleshoot issues by viewing the runtime logs. See View runtime logs for collector runs.
• Discover and evaluate the harvested data assets in the Data Catalog. See Governing the Data Catalog.