Managing data fabric tables
Summarize
Summary of Managing data fabric tables
Data fabric tables in the ServiceNow AI Platform provide a virtual representation of data stored externally or locally, allowing users to access and interact with that data as if it were stored within the ServiceNow instance without duplicating it. Data stewards are responsible for creating and managing these tables, enabling seamless data integration across sources.
Show less
Managing and Viewing Data Fabric Tables
- Data fabric tables can be viewed and searched by label or name, with filtering options by data source, connection, and creator.
- Tables are categorized under Active (from active connections) and Others (from deactivated or unconfigured connections) tabs for easier management.
Primary Keys
- Primary keys identify unique records and enable references between data fabric tables.
- When mapping data, primary keys from the source are automatically defined; if none exist, users can select a unique column to serve as a primary key.
- Primary keys do not enforce constraints on data uniqueness; users must ensure uniqueness themselves.
- Primary keys cannot be removed once defined; changing them requires deleting and recreating the table.
- Without a primary key, references and searches in the data fabric table are not possible.
Reference Tables
- Data fabric tables can reference other data fabric tables or local tables, establishing relationships and maintaining data integrity.
- To reference local tables, administrators must create specific ACL rules granting the datafabric record operation for security.
- Reference columns require the Display attribute to be set to true to show values properly.
- Referenced tables must have a single primary key with unique values matching the data type of the source column.
Table Queries and Data Lifecycle
- Each time a data fabric table is accessed, a fresh query is sent to the external source, and data is cached in memory only while the list or form is open.
- Once closed, the cached data is purged, and subsequent access triggers a new query to ensure up-to-date information.
Limitations
- Data sources must not use case-sensitive table names, as this prevents schema loading.
- Only a single primary key column can be used as a reference key; composite keys are not supported.
- Only tables with one primary key defined can be referenced.
- Additional limitations are documented separately in KB2017730.
Common Actions
- Create: Establish a new data fabric table to retrieve data from internal or external sources.
- Update: Modify data mappings between source and fabric tables as needed.
- Delete: Remove data fabric tables when no longer required.
- View Records: Access current records within a data fabric table.
- Change Connection: Reassign a data fabric table to a different data source.
- Deploy: Transfer data fabric tables between instances using update sets.
Access data virtually from an external source or a local table on your instance using data fabric tables.
Overview of managing data fabric tables
A data fabric table is a virtual representation of data stored in another source, accessible within the ServiceNow AI Platform. It enables users to interact with data from another source as if it's stored locally, without duplicating the data. Data stewards are responsible for creating and managing data fabric tables using the process shown in this infographic.
Viewing data fabric tables
View a list of all the data fabric tables that data stewards have created on the Data fabric tables tab.
- Search for a data fabric table by label or name.
- Filter the list of tables by data source and connection.
- Filter the list of tables by creator.
- View a list of data fabric tables from active connections in the Active tab.
- View a list of data fabric tables from connections that are deactivated or not configured in the Others tab.
Defining primary keys
A primary key is used to indicate uniqueness and identify records in a data fabric table. Designating a column as a primary key enables you to create references to that column from other data fabric tables. When mapping data in a new data fabric table, primary keys from the source table are automatically defined in the target data fabric table. If the source table doesn’t have a primary key, you can create one in the target table by choosing a column with unique values.
Note the following details when defining primary keys:
- Selecting a primary key doesn't enforce primary key constraints on the data fabric table or on the remote system. You must verify that records identified by the primary key are unique.
- Once you define a primary key, it can't be removed. If you determine that you need to change the primary key later, you must delete and recreate the data fabric table first.
- If you decide not to define a primary key when creating a data fabric table, you can do so by editing the table later. Note that without a primary key, you can't create references to this table and searches won't work until a primary key is defined.
- For every primary key column identified in the source table, a corresponding primary key is automatically created in the data fabric table. Once the data fabric table is created, you can't modify the primary keys that were derived from the source table.
- You can't use a reference column as a primary key.
Reference tables
When mapping external source columns to a data fabric table, you can add references to other tables. Establishing references between tables creates a relationship between the two tables and provides data integrity. A data fabric table can reference another data fabric table or a local table on your instance. A local table can also reference a data fabric table.
Note the following requirements when creating references between tables:
- To reference a local table on your instance from a data fabric table, an instance administrator must create an ACL rule with the data_fabric record operation on the local table. For example, to create a reference to the
User [sys_user] table, the following ACL rule is required:
- Type: record
- Operation: data_fabric
- Name: User [sys_user]
- To see values in a reference column when viewing records in the new data fabric table, you must set the Display attribute to true for the referenced column. You can manage this attribute in the referenced table's definition. See Select a field as the table display value.
- Ensure the referenced table has at least one primary key column.
- Ensure the primary key column in the referenced table contains unique values.
- Ensure the primary key column matches the data type of the source column.
Table queries
Records retrieved from a data stored and stored in a data fabric table have a finite duration, or life cycle, within the ServiceNow AI Platform.
After a data fabric table is connected to a data source, each time the data fabric table is viewed or accessed, a new query is made to the data source. The retrieved data lives in memory for as long as the list or form appears. After you close the list or form, that retrieved data is purged from memory. The next time that you use or view the retrieved data in the data fabric table, records are repopulated from the data source.
Data fabric table limitations
- When mapping data, ensure that the data source does not use case-sensitive table names. For example, if both MY_TABLE and my_table exist in the database, the schema will not load because the system cannot differentiate between the tables.
- When creating a reference to another table, you can only select one primary key column as the reference key. You can't select a composite key as the reference key.
- When creating a reference to another table, you can only reference a table that has one primary key defined.