Architectural Choice: SFTP/MID "Pull" vs. REST API "Push" for Asset Discovery Ingestion

Mansor2
Tera Contributor

4 hours ago

Objective: Rapid Asset Visibility (HAM/SAM focus).

Constraints: * Avoiding paid IntegrationHub licenses (Starter/Pro) and third-party/vendor store connectors.

  • Note: The Service Graph Connector for Lansweeper is not an option as it is not currently subscribed and we are avoiding the additional MID Server infrastructure it requires.

  • Goal: Leverage ServiceNow’s native ingestion framework (IRE/RTE) using a serverless approach on the client side.

Scenario: We are integrating an on-premise Asset Discovery tool, Lansweeper, into a global ServiceNow instance. We are weighing two distinct technical approaches:

Option A: The Legacy "Pull" Model (SFTP + MID Server)

  • Architecture: Discovery Tool > Client SFTP Server > MID Server > ServiceNow.

  • Mechanism: Scheduled batch jobs where the MID server "pulls" CSVs from the SFTP.

  • Infrastructure: Requires a dedicated SFTP server (provisioned/patched by Client) and a MID Server.

  • Governance: Relies on legacy Transform Maps and custom scripting for data validation and error handling.

Option B: The Modern "Push" Model (Serverless REST API)

  • Architecture: Discovery Tool > Lightweight Local Script > HTTPS (Port 443) > ServiceNow.

  • Mechanism: Event-driven. A PowerShell/Python script on the source machine "pushes" the data directly to the Import Set API as soon as the scan finishes.

  • Infrastructure: Zero middle-tier servers. No SFTP, no MID Server (eliminating infrastructure maintenance overhead).

  • Governance: Leverages the Robust Transform Engine (RTE) and the IRE via CI Class Manager for high-performance ETL mapping and automated duplicate detection.

Key Questions for the Community:

  1. Security: Is Port 443 (REST) generally preferred by modern CISOs over Port 22 (SFTP) for discovery data in 2026?

  2. Maintenance: Given the goal is Asset Visibility, does the overhead of maintaining an SFTP/MID chain justify the "batch stability" in a modern SaaS-first environment?

  3. Governance: Is leveraging the IRE/RTE (Robust Transform Engine) considered the definitive "Best Practice" over custom legacy Transform Map logic for inventory data loads in 2026?

  4. Timeline: Based on experience, should a focused 8-week sprint (Option B) be sufficient for a technical rollout, or is a 6-month timeline still considered a modern benchmark for a single-source integration?

0 Helpfuls
  • 197 Views
1 REPLY 1

Naveen20
ServiceNow Employee

4 hours ago


Two Options

Option A — Legacy "Pull"
Lansweeper → Client SFTP Server → MID Server → ServiceNow
Scheduled batch jobs. Requires SFTP server + MID Server provisioning.

Option B — Modern "Push"
Lansweeper → PowerShell/Python Script → HTTPS (443) → Import Set API → IRE/RTE → ServiceNow
Event-driven, fires post-scan. Zero middle-tier servers.

---

## Key Questions & Our Assessment

### 1. Security: Port 443 (REST) vs Port 22 (SFTP)?

REST on 443 wins. The push model uses outbound HTTPS only — no inbound firewall rules needed on the client network. You get TLS 1.2/1.3 in transit and OAuth 2.0 for credential management, versus SSH key rotation for SFTP. This aligns with zero-trust principles; the traffic looks like any other SaaS API call. Most CISOs in 2026 would flag a new SFTP dependency as legacy debt.

### 2. Maintenance: Does the SFTP/MID chain justify "batch stability"?

For a unidirectional asset data push — no. The MID Server excels at bidirectional communication (orchestration, live probes, credential-vaulted discovery). For pushing scan results, it's overengineered. You'd be maintaining an SFTP server (OS patching, disk monitoring), a MID Server (JRE updates, upgrade compatibility every family release), and scheduled job orchestration between them. A single script eliminates that entire tier. If it fails, you get an HTTP error code immediately — no ECC queue backlog debugging.

### 3. Governance: IRE/RTE vs Legacy Transform Maps?

This is where Option B is strongest. The IRE with CI Class Manager and RTE is ServiceNow's strategic direction for CMDB/asset ingestion:

- Authoritative source precedence — policy-driven reconciliation when a second source is added later.
- Identification rules — deduplication on serial number, MAC, or composite keys without custom `onBefore` transform scripts.
- RTE bulk processing — significantly better throughput than row-by-row transform map execution at scale.
- CI Class Manager — auditable field mappings instead of logic buried in scripts only the original developer understands.

Legacy Transform Maps still function, but for a net-new integration in 2026, choosing them over IRE/RTE is hard to justify in an architecture review.

### 4. Timeline: 8-Week Sprint or 6-Month Program?

For a single-source push integration, 8 weeks is realistic:

| Weeks | Phase |
|-------|-------|
| 1–2 | Data analysis, IRE identification rule config, CMDB/HAM class mapping |
| 3–4 | Push script + Import Set API staging table + RTE mappings |
| 5–6 | UAT with production-representative data volumes |
| 7–8 | Production cutover, monitoring, documentation |

The 6-month timeline is typical for Option A because you're provisioning infrastructure and navigating firewall change requests. Remove that overhead and the timeline compresses naturally.

Caveat: The 8-week target assumes a stable Lansweeper export schema and a pre-defined CMDB class structure. If upstream data quality work is needed (normalizing manufacturers, reconciling asset tags), budget additional buffer.

---

TL;DR: For a single-source, asset-visibility-focused integration in 2026 with no IntegrationHub licensing, the serverless REST push model (Option B) reduces infrastructure, aligns with modern security posture, and leverages ServiceNow's strategic IRE/RTE framework. The SFTP/MID chain (Option A) remains valid but carries disproportionate overhead for this use case.