IntegrationHub Import: How to filter DataStream content?

felixmiske
Tera Guru

Hi there,

 

Just playing around with the IntegrationHub Import module (All > IntegrationHub > All > IntegrationHub Import) - and I'm a bit confused on how to filter if the Import Source is an Spoke DataStream.

felixmiske_0-1725265302189.png

For example, just selecting Filtering On=Serial Number and writing a String into the Filter Value field for a Serial Number does produce errors during the Import but does not filter for the respective Serial Number.

 

Is there any syntax for field Filter Value? At least the documentation Importing and streaming data in Integration Hub does not mention anything in that regard 🤔 

 

And bonus question: is there any option to have a bit more complex filtering?

Example:

  • do only import rows where Model contains iPhone || Model contains iPad
  • do only import rows where Model does not contains Laptop AND Model does not contain Workstation

 

 

Thx in advance for any helpful tips 🙂

(please do not just reply with a link to the documentation which I have referenced myself above)

 

5 REPLIES 5

Robin J_
Tera Guru

Hi Felix,
If i am not mistaken I did something here but cant find the code at the moment... anyways let's say you have a large JSON file in response on the Data Stream and you want to filter out all fields containing ="Coffee" (bad example but maybe it puts you on the right track)...

Then last time i tested (some months ago) I did just alter the code here:

RobinJ__0-1725266666303.png


So for example if a field called "type" is "coffee" which you don't want you could write

if (outputs.type == "coffee") {
outputs.state = 'SKIP';
}

It was something like that I did when testing. Unfortunately in a lost PDI and in the project for that integration we did not have IntegrationHub licenses so used something else 🙂

Anyways hope it helps some and for your bonus questions I would assume this is solved by coding in the "SKIP's"

Kind Regards,
Robin

Hi @Robin J_,

 

Thx for the reply. I'm aware that I could use a flow and use custom code to filter - but my question is specifically about the IntegrationHub Import module.

 

As the IntegrationHub Import module uses a step-by-step guide without the option to use custom code or a flow / action for filtering, I would like to know how to filter inside of the IntegrationHub Import module guardrails 🙂

Aha, it could be that you quickly met some limitations here 😞 Could be the case. It would absolutely be nice to do some filtering inside those guardrails.

snowArchitech
Tera Contributor

any post where this has been answered?