In this series of articles, we'll cover searching external content with AI Search.
Chances are your organization relies on more than one system to get work done. Knowledge can be scattered across multiple sources. Whether it's an employee having an issue or an agent trying to resolve a customer's case, adding content from other systems to your AI Search experience maximizes the value of your solution.
Adding content from other systems to the AI Search index means:
This introduction covers:
- external content definition
- how external content can be made available for AI Search
- what are the different methods available to get external content
- which one to choose based on your needs
Check out this AI Academy session on how to make that external content searchable in AI Search!
Internal vs External
If you're looking at expanding AI Search to external sources, it probably means that you have a good understanding of indexing content for internal tables (see the quick start guide). So, let's start with a quick refresher of that process.
When internal content is made available to users through AI Search, the following applies:
- There is a table on the Now Platform with data (KB articles, catalogs, users, tasks, etc.)
- This table is "indexed" via an indexed source and then filtered via search sources
- The data is used in an application by AI Search
Overview of the process for internal data
External content is, by definition, content that lives outside of the platform. Therefore, it is not available for applications inside the platform. So, a few additional steps are required:
- Make that content discoverable, we call this "ingestion"
- Interpret the content into a language that the Now Platform and AI Search understand, with an "external content schema" table
- This table can now be used by an indexed source and then filtered via search sources
In many ways, this follows the process of building an integration with another system. The difference is that data isn't duplicated in the platform. Instead, columns serve as a map to content used by the AI Search index. When a user selects an external document search result, they are directed to the source document in its original location.
Overview of the process for external content
Just like an integration, there are two main factors that influence the design of this solution: how and when is the data updated.
How to ingest external content
There are different methods available to ingest external content and make it available in your applications using AI Search.
Search connector application
The first method is to use a Search connector application. These pre-built applications available in the ServiceNow Store provide the tools to index content from specific sources and use cases. Sometimes, these solutions can also provide the external content schema table.
This is the preferred option if you're looking for a turnkey solution with little implementation effort. Connectors are built by ServiceNow or partners for specific sources and use cases, meaning that a connector might not be available for all systems. Also, note that customization options are limited, and the provider of that connector controls the update cycle and content security options.
ServiceNow provides a SharePoint Online Search Connector in the app store. Refer to the docs under the Supporting Links and Docs section.
Building an integration
The other two methods involve building the integration.
The AI Search spoke leverages the Flow Designer interface. It contains Flow Actions that can be used to jumpstart the integration build in Flow designer. It is available through an Integration Hub subscription.
This spoke is documented in the AI Search spoke docs and leverages the External Content REST API.
The External Content REST API provides endpoints you can use to POST from external systems. This method is the most complex, but it offers the most flexibility. This API is documented in the External Content Ingestion API docs or via the REST API Explorer in an instance under AIS External Content Ingestion API.
For an example of how to use this API, see the How to index external content with AI Search article on Community!
Both these methods support content security. For more information on security, refer to the docs External content security for AI Search.
When to ingest external content
There are different ways to schedule the update of the external content.
Scheduled Batch
The AI Search spoke comes with batcher actions that allow processing multiple records in a batch. The trigger is a schedule that can run daily or more frequently based on the needs. This can be done via the AI Search spoke actions in Flow Designer or a custom scripted solution.
Stream or On-demand
A stream or on-demand approach is used to process only small chunks of data when records are created, updated or deleted in the source system. This can be done via the external system, middleware or a webhook mechanism in the same fashion as an integration.
What's next?
Adding external content to AI Search allows users to get faster and better results across multiple systems. Scattered knowledge sources can be connected in a unified experience via pre-built search connectors, Flow designer actions and APIs.
References
 |
Documentation |
 |
Academy |
Check out the AI & Intelligence forum to get the best content about ServiceNow's AI products!
Any specific instruction or guidance on how to create the User Mapping for the SharePoint Online Search Connector (link: Setting up the SharePoint Online Search Connector (servicenow.com))? There's a default table that is created for this (screenshot below), but documentation doesn't state where/how this is populated. There are also a few docs/dev sites for external user mapping, but nothing specifically on how to set these for SharePoint.
