Delta imports in LDAP

KG875227053
Tera Contributor

 

Hi Team,

We are planning to evaluate the feasibility of implementing a delta synchronization approach for Active Directory (AD) group imports into ServiceNow.

Currently, the Group OU Definition performs a full import using the LDAP filter:

(objectClass=group)

As a result, every scheduled synchronization queries and imports all groups from Active Directory, regardless of whether any changes have occurred since the previous run.

We would like to understand whether a delta synchronization approach can be implemented to import only newly created, modified, or deleted groups, thereby reducing the overall load and improving synchronization efficiency.

If anyone has previously implemented a similar solution or has experience with this approach, please share your insights on its feasibility, recommended design, and any potential challenges or limitations.

Thank you.

 
1 REPLY 1

kmohammed
Tera Guru

Hi,

 

Yes, it is feasible for newly created and modified AD groups - but not as a complete replacement for the full import, because deletions cannot be detected this way. The workable design is delta for day-to-day runs plus the full import kept as a periodic reconciliation.

 

How the delta works:

1. AD stamps every group with whenChanged, and group membership edits also update the group's whenChanged (member is an attribute of the group object), so changes you care about do move the timestamp.

2. Change the Group OU filter from just (objectClass=group) to also require whenChanged >= your watermark, in AD's format, for example:

(&(objectClass=group)(whenChanged>=20261001000000.0Z))

The watermark is the start time of the last successful delta run, stored somewhere the import can update - a system property or a small watermark record.

3. The LDAP filter on an OU definition is static text, so something has to rewrite it before each run: a scheduled job/flow step before the scheduled import that writes the current watermark into the OU definition filter, runs the import, and only advances the watermark on success. Use the same transform map as the full import, coalescing on the AD objectGUID/source identifier, so delta rows update the existing group instead of creating duplicates.

 

Limitations and challenges - these are the parts to design around:

- Deletions: a deleted group returns in NO query, including a whenChanged query. Out-of-the-box LDAP import does not query the AD Deleted Objects container / DirSync either, so a delta-only setup will keep deleted groups forever. That is why you still need a periodic (for example weekly) full run, plus a reconciliation step after it - the transform by itself does not deactivate records that were missing from the load, so compare last-seen/source timestamps and deactivate groups the full load no longer returns.

- whenChanged is NOT a replicated attribute: each domain controller holds its own value. Pin your LDAP server to one DC for these queries, or your watermark becomes meaningless when the query lands on a DC that has not replicated the change yet. Replication lag and clock skew are also why you should overlap the window - e.g. query from watermark minus 10-15 minutes. Re-processing a few groups is harmless because of the coalesce.

- Group nesting/member-of on the user side: if your design also relies on user records' memberOf, remember that is maintained by AD, not written by the group object - keep the user import logic unchanged and test membership changes end to end in the delta.

 

Net result: the delta run will only pull groups created or changed since the watermark, which removes most of the load, and the scheduled full import + stale-group reconciliation covers the delete case the delta cannot see.