This blog post is #5 - and the last - in a series of posts where I show you how to migrate from a more traditionally implemented integration using scripted web services to a modern Flow and Integration Hub based one. In this post I will summarise what we have achieved and what the benefits are including some resources to take this further.
If you have not yet seen the other blog posts, please check them out here:
- Blog 5: This blog about Summary and Benefits
Summary of achievement
Over these series of blog posts I have shown you how to replace a scripted - i.e. traditional - implementation of an integration to a more modern low code approach. I used the OMDb API which is publicly and freely available as an example. This included two different endpoints to search for a movie and retrieve detailed information about a given movie.
We replaced scripted business rules with modern Flow Designer based approach leveraging a newly created custom spoke for Integration Hub.
Benefits
Obviously, we at ServiceNow do believe these new technologies are a significant improvement over our legacy options, and yes we do ask for a fee to use Integration Hub to its full potential. So what are the benefits for you when using it?
Low Code
As a professional developer you might argue that Low Code as such is not a criteria of importance to you. And yes, you are probably not faster in building the initial version of an integration wether you use a well-known scripted approach vs. something new and somewhat strange thing like Low Code.
I hope though that this blog series at least showed that the effort in building an integration is comparable for a simple integration like the OMDb one shown. Creating a new spoke and putting the web service calls in Actions within Flow Designer is not more complex than writing scripts. Would I recommend that this part should be done by lesser experienced developers or even Citizen Developers? No, I don't think so. To build a resilient integration I still believe you need a good level of expertise in things like error handling and data mapping. Hence building out a spoke is still a task in the hands of professional developers. Once the spoke actions are available though, I do believe making these actions available to a Citizen Developer building out a business process using Flow Designer is becoming an option and much simpler than explaining someone how to develop and or use a script for the same activity.
So overall, using low code tools opens new possibilities not available with purely scripted implementations.
Maintenance
Similarly to Low Code a professional developer might say that it is much easier for him or her to adapt a piece of code when time comes. But be honest, how often have you looked back at a piece of code you have written say 6 months ago and immediately knew what it does? Now consider it was not even your piece of code. Sure, we all have coding guidelines which should make this step better, like comment your code, name variables meaningfully etc. How well does it work in real life?
Using a Low Code approach makes reading and understanding of implementations from months ago or from other developers much simpler. With a proper developed spoke, changing how the spoke operates will not impact the processes using the spoke actions. There are by default defined inputs and outputs decoupling different elements in the platform.
It is also much easier to find where a given spoke action is used. There is a function in Flow Designer called 'See related flows' showing you exactly where your action is being used:
The same feature is available for subflows.
Try finding all the places someone could hide a piece of script to call your api.
Credentials & Connections
All integrations need credentials of some shape or form. With scripted API's these credentials tend to be buried in lines of code. If you have implemented a few integrations you probably build your own best practice of maintaining these elements outside code like in properties or lookup tables.
With spokes all credentials are maintained securely in a central place within Flow Designer.
In addition, these Connection and Credentials are not deployed using Update Sets or App Repo. They are intentionally marked as data and not configuration items. Most likely the connection and credentials will differ from development to test and production environments. Excluding them from deployment removes the risk of accidentialy taking to the wrong endpoints. We also have default exclusion rules and data preservers in order to prevent these data elements from cloning back. This means after clone, your test instance will still point to test environments of other systems.
No more manual work to reconfigure integrations post clone!
Security
Another big point for a lot of companies is security around who can use certain integrations. Assume you want an integration to your core HR system to update user profile information. This can be a pretty critical interface which you do not want anyone on your developer community (e.g. Citizen Developers) to just use. With spokes you get a lot of controls to allow access the these actions or not.
Content Filtering Rules allow an administrator to define who can see certain spoke actions and who cannot - preventing unauthorised usage.
The subflows using spoke actions can be set to run in the user's context or with explicit roles. All scripted integrations usually leverage a system account to execute having way more access then the user would have.
Trouble Shooting & Error Handling
When an integration fails for some reason, finding the culprit can be really hard. In a traditional implementation you will most likely find yourself in staging test data and running them over and over again potentially looking at a Script Debugger window to follow the data traces trying to understand where a wrong data content has been passed in.
With Flow Designer it is easy to activate Flow Reporting for a bespoke flow - or a set of flows - which automatically generates a massive execution stack. You can quickly see all data elements, all decisions the flow made etc.
This makes debugging a thing from the past.
All elements in Flow Designer have built in error handling possibilities. From Try-Catch logic to automatic retries at integration steps.
This allows from a very granular control at specific integration steps up to complex error handling routines at process level.
Monitoring
Monitoring is a crucial part for any integration. Most of the times an API based integration will work just fine, and that is why we spend so much time at implementation stage to make it resilient. But what happens on the day things break? I already mentioned the debugging and trouble shooting part, to get there we need to know something went wrong.
Integration Hub comes with a dashboard delivering lots of information about all your integrations passing through it.
Here is one screenshot providing an overview of the load passing through Integration Hub, it will allow to drill down into specific spokes or actions, days of a week, protocols used etc. Another metric is successful vs. failing transactions.
The number of transactions is an important metric as ServiceNow charges for Integration Hub usage based on it. Some customers and partners are concered about this claiming it is an intransparent or unsafe pricing model. While I understand this concern, I believe it is one of the pricing models more transparent and clearly defined than in other places of the platform. The packages sold contain in the smallest payed SKU already 1 million transactions per year - this equals to 2 transaction every minute, 24x7x365. When you actually reach this volume I would bet you can easily show the value of Integration Hub as it has helped automate lots of previously manual work.
Pre-built spokes
While the whole blog series is focused on building a custom spoke to replace a scripted API, before building a custom spoke you should check the list of available spokes and see if there is something available already. Chances are high that with 170+ spokes available - all in the same licence package, no extra fees - there is one matching your need. ServiceNow delivers spokes in basically two flavours:
- most spokes can be seen as a toolset providing access to API functions of 3rd party service via Actions in Flow Designer. You will still need to build out the process layer on top to make it do what you need. A good example is the Jira spoke, once installed it will do nothing until you start implementing the needed flows to trigger data exchange to Jira. But it does provide a set of actions like Create Issue, Upload Attachment etc. speeding up your implementation significantly.
- some spokes come with fully fletched out processes, e.g. Password Reset for AD. In this case the plugin will bring all needed functionality as a kind of turn-key solution.
If there is no spoke available from ServiceNow, make sure to also check the [ServiceNow Store, partners do provide spokes for additional applications. Note: These might incur an extra licence fee.
You can also find the the list of spokes and the packaging information in this document.
Conclusion
Thanks for going though all this with me. I hope you find the information useful and maybe you learned a bit or two. Please use the comment function to highlight things I missed, points you agree or disagree with me. I am happy to revise, add or at least discuss where needed. If you have some great ideas what our Integration Hub product should also do, do not hesitate to leverage our Idea Portal to submit them. Our product managers are reading these very carefully.
1 Comment
