In this blog, I’m excited to share some insights and proof of concepts around a technology that’s been making waves in the consumer space: Augmented Reality. In this blog covering a POC I've named the Augmented Reality Assistant (ARA) for ServiceNow ITSM, I’ll cover:
If you would like to see this technology formally implemented or productized, your feedback to the questions at the end of the blog is greatly appreciated. Whether you comment that you see value or that all you see is an overhyped technology not yet mature enough for the enterprise, your feedback is appreciated. Many thanks! Darius, ITSM Product Management.
I. Augmented Reality Overview
Merriam-Webster refers to Augmented Reality as “an enhanced version of reality created by the use of technology to overlay digital information on an image of something being viewed through a device”.
Augmented reality technology will be pivotal to reducing the friction between people, process, and technology. If we think about the commonly used hardware devices and interfaces in use today, from mobile devices to keyboards, monitors, and mouse’s, they are merely interfaces from which we provide inputs to software to complete an action or lookup information.
So why not optimize on this process by removing the need to use your hands to interface with the hardware entirely, while providing contextual information without needing to manually do lengthy searches? Visually overlaid augmented reality provides this seamless solution.
Augmented reality is an example of a technology that has already made waves on the consumer side of the world with the now infamous Niantic's ‘Pokemon GO’ game. If history is any teacher, it’s only a matter of time until we see similar experiences being adopted within the enterprise. In doing so, we will improve the fulfiller & manager experiences to more efficiently complete work and make business decisions. If you look for it, you will see that there are already successful implementations of the technology in a business setting, especially around providing contextual knowledge to technicians.
II. Augmented Reality Technology Providers
The rising interest and demand for the technology can be seen through the investments made by consumer centric organizations into developing augmented reality, notably:
Apple has a ARKit which can be accessed on the newer iPhones and leveraged by app developers.
Google’s Tango, while currently only for specific devices, is available to app developers.
In addition to the large corporate players looking to leave their mark on this space, smaller open source projects are also readily available. These include, but are not limited to:
Proprietary:
•Layar SDK [iOS and Android]
•Catchoom CraftAR AR SDK [iOS and Android]
•Vuforia Augmented Reality SDK
•Wikitude SDK –first AR SDK providing JavaScript API to work with augmented reality experiences
Open Source:
•Arma – OpenCV-based
•ARToolKit – C-library
•Augment – Tablets & smartphones
•JavaCV –Java/Android interface
•DroidAR – Android AR framework
Beyond the software, we must also consider the hardware required for the current augmented reality implementations. There are quite a few major vendors already working on the wearable versions of the technology, most notably including Microsoft’s Hololense, the Google Glass Enterprise Edition, and the more than billion dollar funded Magic Leap. Many smaller vendors like iGlass AR, Vuzix Blade AR, and others have already developed consumer/business grade models. Future advances and research (e.g. neuralink) are promising more efficient and less bulky hardware integrated directly into our bodies.
III. Augmented Reality ITSM Proof of Concepts
In our augmented reality proof of concepts, we decided to use readily available hardware, using both Google Glass & Microsoft HoloLens to mock-up a few use cases. Examples and descriptions of the exact use cases built per device can be found below:
Google Glass:
The Google Glass proved lightweight and easy to use, however the small screen size and limited interactions meant it could only handle basic use cases that had linear process flows without significant information. In this case, we used the Explorer edition as it was readily available. The app is written using Java / Android SDK.
Maintenance List: Provides an easily accessible list of field service maintenance requests and their associated maintenance tasks. Technicians can then easily progress the state of these tasks and leave work notes on the go. A similar concept can also be applied to change requests and their child change tasks. In the demo below, a tech drills into a carpet cleaning request, and completes the task to rent or call a carpet cleaner after leaving work notes indicating the job was done.
As we can see, the lifecycle of the task, including the added work notes, are tracked in the history.
Incident Creation: Technicians can quickly create an incident using a picture taken using the google glass, and have the ability to use voice to set short description quickly. This helps reduce intake processing and quickly provides an incident with rich information. In the demo, a tech takes a picture of the device having the issue, and narrates the short description to trigger incident creation.
As observed in the screenshot below, the created incident contains the identified caller, short description, and attached image.
Barcode Scan: The Barcode scan allows a tech to quickly scan an asset barcode to gain device information and to create an incident, if necessary. Here, a QRcode is scanned to retrieve asset details. The tech then has the ability to quickly jump into a create incident flow.
In the screenshot below, it can be observed that the affected CI is automatically set on the created incident.
Microsoft HoloLens:
The Microsoft HoloLens allowed for a much large field of view compared to the google glass. In addition, it offers the ability to interact with objects in space with gestures in air. This meant use cases that required more interaction and display real-estate were best tested on this device. We used the developer edition in our testing, and the app created is done in C# / Unity.
QR Code Lookup: Scanning a QR code quickly pulls up relevant information, such as configuration item information. This allows techs working with end user devices to quickly gain context around a device by simply pulling it’s barcode/QR code into the field of view. The scanning automatically reads the barcode and identifies the relevant form of information.
Escalation via Skype Call: An integrated skype call allows a technician to seamlessly escalate to a higher tier technician via live audio and video to be sent into and out of the device. This means the higher-level technician can see the issues directly from the perspective of the tech, and the tech can see information and hear insights sent by the higher-level technician. In the demo, a tech triggers an escalation, and narrates the name of the tech to escalate to, which triggers a skype call to the user.
Knowledge Search: Since the HoloLens also offers voice based commands via Cortana, a voice based knowledge search can be enhanced by image search. The larger screen size relative to the Google Glass means that onscreen knowledge content can be displayed in its entirety, while also allowing users to navigate search results more easily. In the demo, a tech selects a troubleshooting KB article from a returned list, which opens the article in a new window.
IV. Other ITSM Use Cases we’ve identified for wearable augmented reality:
Caller Identification: Utilizes facial recognition of an employee and identifies their currently opened incidents and requests. Useful in a walk-up setting.
Assisted Fulfillment for technicians: where the wearable scans an asset tag and lets the technician create an incident, create a change, or consume the asset for a given user. Selections can be done visually in space or through voice commands.
Integrated Knowledge Search: Computer Vision can identify an object and bring up relevant knowledge base articles. Alternatively, a technician can use voice to search a knowledge base. Knowledge Article information is presented overlaid for contextual insights, while allowing the technician to retain full mobility of both hands. AR enabled KB articles can actually provide overlaid visualizations, as opposed to traditional plain text.
Integrated ITSM Information: Provides any outstanding Change Request and Problem information for a given visually identified asset (either through scanned barcode or visual recognition on the asset class), to aide in RCA, prevent unplanned outages against CI’s and quickly link workaround information from a Problem.
Business Service Map Data Visualization: Visualize and traverse a business service map using gestures and voice commands to easily drill into dependencies, identify changes, problems, and outages overlaid on the nodes visually.
Report Visualization: Place reports anywhere in space, such as on the walls of an office, and quickly drill into data.
V. Now that you’re thinking about augmented reality use cases for ITSM:
- Has your organization explored the use of Augmented Reality in your business?
- Would your organization be interested in piloting augmented reality?
- Do you have a use case you believe Augmented Reality would help make more efficient or solve? It can be new or an existing use case mentioned above.
Let me know in the comments below!
7 Comments
