Data Stream Errors on Empty Rest Response

[Brian G]

Currently I'm creating a flow that has 2 API steps with the goal of pulling all Service Accounts  for our population of projects:

1. First step: Identify Project population aka get all projects. An API call to pull a list of all projects from google cloud

2. Since Service Accounts are housed under Projects in a 0-many fashion, Another API call is needed for Each project pulled in step 1. This rest call pulls each Project record's Service Accounts data.

This step requires pagination to be used so a Datastream is being used for this.

I have this partially working, but am running into a rather puzzling issue.

if a project has accounts under it the ServiceAccounts API response looks like:

{
"accounts": [
{
"name": "projects/prj-infrastructure-global-001/serviceAccounts/sa-grafana-sandbox@prj-infrastructure-global-001.iam.gserviceaccount.com",
"projectId": "prj-infrastructure-global-001",
"uniqueId": "102034640071361008578",
"email": "sa-grafana-sandbox@prj-infrastructure-global-001.iam.gserviceaccount.com",
"displayName": "sa-grafana-sandbox",
"etag": "MDEwMjE5MjA=",
"description": "reads metrics from cloud monitor",
"oauth2ClientId": "102034640071361008578"
},...etc

]

However, if a project doesn't have any accounts under it the API response body will be empty: {}

I have my splitter step configured to look for $.accounts, but this fails with "Could not find path in stream: $.accounts" when i hit project with no accounts and the response is empty.

There does not seem to be a way to gracefully ignore these failures and move on and no scripting is available between the REST step and the splitter step in these data streams which seems like an oversight for this type of issue.

What options do I have to code around a empty response situation like the above?

[Brian G]

Attaching DS1-9 showing on screen setup of Datasource

[Brian G]

See Attached Screens of Datastream