LDAP import missing values

[bennyphipps]

Hi,

we've got a LDAP data source setup (using a MID Server) which is only pulling in 6000 out of 7000 records.   I can see AD groups that are virtually identical in name and all other attributes (except the obvious ones like ObjectSID, ObjectGUID samAccountname etc) some which are included and others which are not there's no fields with massive values in either etc.

This is not a problem with Transform map as the data is missing from the import itself.   I've tried clearing down the Filter in the OU Definition and that made no difference either.

I can view the AD Groups from the instance when I use the LDAP Browse functionality ok though.

Can anyone help or suggest anything?   My import table field sizes are all pretty large to account for truncation etc but some of the groups.

Cheers,

Ben

[bennyphipps]

Ok cool no worries... I have rasied a HI request anyway.

So it's all very weird but this is teh filters I used and the different

(objectclass=group) - ~6000 records.

(|(sAMAccountName=*)(sAMAccountName=DATA*)) - ~6000 records.

(|(sAMAccountName=*)(sAMAccountName=DATA*)(sAMAccountName=DATA A*)) - 6800+ records.

Which seems odd to me.

Regards,

Ben

[poyntzj]

couple more

if you try without the (sAMAccountName=*)

what if you try with

(|(sAMAccountName=*DATA*))

Got to say your findings look really odd to me.

I would expect

sAMAccountName = DATA* to bring back anything group that starts with DATA

sAMAccountNme = *DATA* to bring back anything with DATA in it.

objectClass=group to bring back anywhere the class is group, regardless of name - and should not need the sAMAccountName=*

just looking at this

http://social.technet.microsoft.com/wiki/contents/articles/5392.active-directory-ldap-syntax-filters...

Could try using the objectCategory and see if that makes a difference.

[bennyphipps]

6094 seems to be teh magic number of records I get back (that's what I got when I used object category)

It seems there's certain Objects that need to be specifically asked for in order for them to appear...

Will disucss further with ServiceNow directly but thanks for your help Julian (I'll update as to what happened)

............... hopefully it'll just be user error of some kind.

[poyntzj]

I will be interested to see the findings too.

as a "workaround", you could create a few imports

(|(sAMAccountName=A*)(sAMAccountName=B*)(sAMAccountName=C*)(sAMAccountName=D*))

(|(sAMAccountName=E*)(sAMAccountName=F*)(sAMAccountName=G*)(sAMAccountName=H*))

and so on and schedule them

[bennyphipps]

So I thought I'd check the MID server logs... and (I don't know much about this at all tbh) but it's giving results like this below (spawned from a filter of (objectClass=group):

10/22/14 17:08:40 (268) Probe: LDAPProbe:a9b5cab76fb751001b2d2c05eb3ee421 DEBUG: LDAP target is ready

10/22/14 17:08:40 (268) Probe: LDAPProbe:a9b5cab76fb751001b2d2c05eb3ee421 LDAP Processing RDN OU=Data and range

10/22/14 17:08:41 (282) Probe: LDAPProbe:a9b5cab76fb751001b2d2c05eb3ee421 Enqueuing: C:\agent Prod\work\monitors\ECCSender\output_s\ecc_queue.149389e08330000001.xml

10/22/14 17:08:41 (828) Probe: LDAPProbe:a9b5cab76fb751001b2d2c05eb3ee421 Enqueuing: C:\agent Prod\work\monitors\ECCSender\output_s\ecc_queue.149389e0a550000001.xml

10/22/14 17:08:41 (891) Probe: LDAPProbe:a9b5cab76fb751001b2d2c05eb3ee421 Enqueuing: C:\agent Prod\work\monitors\ECCSender\output_s\ecc_queue.149389e0a930000001.xml

10/22/14 17:08:41 (938) Probe: LDAPProbe:a9b5cab76fb751001b2d2c05eb3ee421 Enqueuing: C:\agent Prod\work\monitors\ECCSender\output_s\ecc_queue.149389e0ac20000001.xml

10/22/14 17:08:41 (969) Probe: LDAPProbe:a9b5cab76fb751001b2d2c05eb3ee421 Enqueuing: C:\agent Prod\work\monitors\ECCSender\output_s\ecc_queue.149389e0ae10000001.xml

10/22/14 17:08:41 (984) Probe: LDAPProbe:a9b5cab76fb751001b2d2c05eb3ee421 LDAP Processing RDN OU=Data and range (sAMAccountName<=_)

10/22/14 17:08:41 (984) ECCSender.1 Sending ecc_queue.149389e08330000001.xml

10/22/14 17:08:42 (203) Probe: LDAPProbe:a9b5cab76fb751001b2d2c05eb3ee421 LDAP Processing RDN OU=Data and range (sAMAccountName>=z)

10/22/14 17:08:42 (437) Probe: LDAPProbe:a9b5cab76fb751001b2d2c05eb3ee421 LDAP Processing RDN OU=Data and range (&((sAMAccountName>=_)(sAMAccountName<=0)))

10/22/14 17:08:42 (671) Probe: LDAPProbe:a9b5cab76fb751001b2d2c05eb3ee421 LDAP Processing RDN OU=Data and range (&((sAMAccountName>=0)(sAMAccountName<=1)))

10/22/14 17:08:42 (905) Probe: LDAPProbe:a9b5cab76fb751001b2d2c05eb3ee421 LDAP Processing RDN OU=Data and range (&((sAMAccountName>=1)(sAMAccountName<=2)))

10/22/14 17:08:43 (123) Probe: LDAPProbe:a9b5cab76fb751001b2d2c05eb3ee421 LDAP Processing RDN OU=Data and range (&((sAMAccountName>=2)(sAMAccountName<=3)))

10/22/14 17:08:43 (186) ECCSender.1 Sending ecc_queue.149389e0a550000001.xml

10/22/14 17:08:43 (373) Probe: LDAPProbe:a9b5cab76fb751001b2d2c05eb3ee421 LDAP Processing RDN OU=Data and range (&((sAMAccountName>=3)(sAMAccountName<=4)))

10/22/14 17:08:43 (607) Probe: LDAPProbe:a9b5cab76fb751001b2d2c05eb3ee421 LDAP Processing RDN OU=Data and range (&((sAMAccountName>=4)(sAMAccountName<=5)))

10/22/14 17:08:43 (825) Probe: LDAPProbe:a9b5cab76fb751001b2d2c05eb3ee421 LDAP Processing RDN OU=Data and range (&((sAMAccountName>=5)(sAMAccountName<=6)))

10/22/14 17:08:43 (888) StatusMonitor.600 Enqueuing: C:\agent Prod\work\monitors\ECCSender\output\ecc_queue.be9f6123343321001ea223665563bc8c.xml

10/22/14 17:08:44 (059) Probe: LDAPProbe:a9b5cab76fb751001b2d2c05eb3ee421 LDAP Processing RDN OU=Data and range (&((sAMAccountName>=6)(sAMAccountName<=7)))

10/22/14 17:08:44 (293) Probe: LDAPProbe:a9b5cab76fb751001b2d2c05eb3ee421 LDAP Processing RDN OU=Data and range (&((sAMAccountName>=7)(sAMAccountName<=8)))

10/22/14 17:08:44 (527) Probe: LDAPProbe:a9b5cab76fb751001b2d2c05eb3ee421 LDAP Processing RDN OU=Data and range (&((sAMAccountName>=8)(sAMAccountName<=9)))

10/22/14 17:08:44 (558) ECCSender.1 Sending ecc_queue.149389e0a930000001.xml

10/22/14 17:08:44 (746) Probe: LDAPProbe:a9b5cab76fb751001b2d2c05eb3ee421 LDAP Processing RDN OU=Data and range (&((sAMAccountName>=9)(sAMAccountName<=a)))

10/22/14 17:08:44 (980) Probe: LDAPProbe:a9b5cab76fb751001b2d2c05eb3ee421 LDAP Processing RDN OU=Data and range (&((sAMAccountName>=a)(sAMAccountName<=b)))

10/22/14 17:08:45 (229) Probe: LDAPProbe:a9b5cab76fb751001b2d2c05eb3ee421 LDAP Processing RDN OU=Data and range (&((sAMAccountName>=b)(sAMAccountName<=c)))

10/22/14 17:08:45 (463) Probe: LDAPProbe:a9b5cab76fb751001b2d2c05eb3ee421 LDAP Processing RDN OU=Data and range (&((sAMAccountName>=c)(sAMAccountName<=d)))

10/22/14 17:08:45 (510) ECCSender.1 Sending ecc_queue.149389e0ac20000001.xml

10/22/14 17:08:45 (682) Probe: LDAPProbe:a9b5cab76fb751001b2d2c05eb3ee421 LDAP Processing RDN OU=Data and range (&((sAMAccountName>=d)(sAMAccountName<=e)))

and so on.....

I wonder if this means anything???

There are probably a few workarounds where I create an extract of AD and export that in or multiple data sources but I wanna get it understood / fixed properly really.