Process mining 101

Want to analyze and troubleshoot core business processes? There’s an app for that

business process mining
  • Process mining can help clarify objectives for corporate digital transformation initiatives
  • Process mining algorithms use data from system event logs to analyze and build models of business processes
  • The models show bottlenecks and redundancies or anything else that slows down a specific process

Manifestos usually bring to mind political bomb throwers or ego-mad artists. But in 2011, a group of nerdy engineers published what they considered a revolutionary document that defined the principles of a new discipline in business intelligence.

Called The Process Mining Manifesto and released by the Institute of Electrical and Electronics Engineers (IEEE), the paper argued that by using techniques in data mining and business-process modeling, managers could peer into the digital guts of their organization to monitor core processes, find faults, and improve them.

Think of process mining as a kind of MRI for the enterprise. Because every facet of modern IT infrastructure generates event log data, capturing interactions between humans and bots on the system, process mining can analyze those event logs to create rich models of real-time system functions.

The emergence of process-mining software in recent years is one reason why the process analytics market is expected to grow from $185.3 million in 2018 to $1.4 billion globally by 2023, according to a recent ResearchandMarkets study.

“It is a very valuable technology that is also very generic,” says Wils van der Aalst, a Dutch computer scientist who is known as the godfather of process mining. “It can be applied to hospitals, governments, airports, production companies, anything.”

Digital transformation: the prequel

Data scientists like van der Aalst have been studying process mining since the early 1990s. There wasn’t a lot of data to work with back then, so no one paid much attention. But now that so many business processes are digitized and throwing off huge volumes of data, “many organizations feel they need to do something with that data,” says van der Aalst.

Often, they’re not sure what. Under pressure to launch digital transformation initiatives to boost productivity, reduce costs, and improve customer service, many executives end up throwing money blindly at these projects with little understanding of the underlying processes they are intended to improve.

In one recent study, 45% of executives said they had no idea where to even start on digital transformation initiatives. Nearly two thirds said they set transformation KPIs without determining what the actual problems are in their core operations. Unsurprisingly, most digital transformation initiatives ultimately fail.

As the digital and physical worlds converge, process mining can help smooth the transition. It can even make trains and planes run on time.

Many companies would improve their odds if they first examined their core business processes to see how efficiently (or not) they get things done. Enter process mining: “If you’re responsible for large numbers of existing processes, you owe it to yourself” to look into using these tools, says Paul Harmon, a business management expert on the IEEE’s Process Mining Task Force and an analyst with BPTrends Associates.

How process mining works

As process mining software analyzes data, it constructs a “digital twin” of myriad IT processes. The mining tools can target any system—for example, enterprise resource planning (ERP), customer relationship management (CRM), or IT services. Any system that generates event logs can be analyzed and “twinned” through process mining. With these virtual clones, everyone from data scientists to CIOs can troubleshoot workflows, automate repetitive tasks, and boost overall efficiency.

Some of the earliest adopters of process mining have been manufacturers— many in Germany and other parts of northern Europe where van der Aalst’s ideas have taken root. For example, the German industrial giant Siemens employs it to troubleshoot its customer processing systems.

As the digital and physical worlds converge, process mining can help smooth the transition. It can even make trains and planes run on time. Consider a baggage handling company that operates in hundreds of airports around the world. It might move some 4 billion pieces of luggage a year. At a typical airport, its systems might log a million daily events to make sure everyone gets the bag they came with.

To the traveler, the process seems frictionless, as it’s meant to. You check your bag at the counter. It’s scanned, routed to storage, eventually loaded onto a cart, and arrives at your plane at exactly the right moment. So why do some bags make it and others don’t? To understand where your suitcase is, or isn’t, you must understand how event logs work.

Every event must have at least three attributes. The first is a case identifier. It could be an online customer’s name or an ID tag for a suitcase. The second is a description of the activity or what happened. In the case of the luggage, it would be baggage check-in. The third is a time noting when the activity took place. Other key data includes the name of the employee handling the bag, the trolley it travelled along, the flight number and destination.

Process mining algorithms start by mapping the current process. The algorithm then generates a process model. The models show bottlenecks and redundancies, outliers, non-compliant activities, and anything else that slows productivity.

“You can drill down, analyze all deviations, see what they have in common and say that’s costing me money,” says van der Aalst. Companies can also generate models that reveal the most efficient groupings of processes that managers can strive to implement.

The digital twins can then be used to game out different baggage-handling scenarios and make predictions. For instance, what would happen if you put two baggage handlers on the back-end conveyor belt, or if the trollies lined over in one area instead of another? How does a rainy day affect the movement of the trollies outside? And so on. By using process mining algorithms to find pinch points and eliminate them, a company could boost productivity by upwards of 20%.

What’s ahead for process mining

As more digital events are recorded in IT systems, executives will have even more opportunities to identify pain points in their processes and resolve them. Van der Aalst isn’t surprised that his manifesto is winning over new converts.

“For a long time, very few people were working in it,” he says. “But if you look at the BPM (Business Process Management) conferences over the past five years, half of the research is now on process mining.”

In short: The process revolution has arrived and will be mined.