Banking on DevOps

Financial services companies are learning how to ship better code, faster

DevOps in banking
  • DevOps is helping financial services companies deploy code much faster in a highly regulated industry.
  • Other benefits include improved software quality, reduced deployment failures, and faster system-recovery times.
  • More than 60% of companies in the sector are expected to convert to DevOps over the next two years.

As CEO of OnlineLoans, a portal that connects consumers to dozens of banks for personal and small business loans, Will Gendron relies on a DevOps-driven IT group to keep pace with constant product integrations and code releases.

Gendron also needs bank partners to have the same kind of IT agility, but that isn’t always the case, especially with older banks. Gendron says he’s had to scrap several deals “as soon as the conversation shifts to implementation. We’ve been able to make the business case, but the [older] banks’ technical teams are often the biggest obstacle.”

They certainly don’t mean to be. IT organizations in the finance sector have been notoriously change-resistant when it comes to new technology, as well as the means to implement it. That’s not for lack of talent or desire, but because of challenging operational constraints.

Financial services firms are subject to a dizzying array of regulatory and security requirements. Every snippet of code released to mobile or online bank customers must pass security, stability, and compliance requirements. Customers, meanwhile, expect the same kind of concierge-grade experience they enjoy with their favorite consumer apps. It’s a challenge many companies have struggled with for years.

Fintech startups are providing further proof for established players that they need to adopt DevOps to go forward.

Enter DevOps, the engineering methodology that has upended software development over the last decade and is finally gaining wider adoption in banking and financial services. A successor to Agile development, DevOps unites developers with operations and testing teams. Nearly a third of IT organizations in the sector have adopted DevOps practices. Another 61% are expected to make the switch within the next two years, according to research by Claranet.

“Business models are changing rapidly in the sector, especially within consumer and small business lending,” Gendron says. “The pace of change is only possible with dev and ops teams working in tandem.”

Chris Pope, global VP of innovation at ServiceNow, sees DevOps almost as an industry mandate. “Established firms in financial services are dealing with massive pressures due to huge regulatory changes happening across the sector,” he says. “Fintech startups are providing further proof for established players that they need to adopt DevOps to go forward.”

Embrace the chaos monkey

While many think of DevOps primarily as a way to deploy new applications faster, some of the lesser-known benefits are especially important in the finance sector. Those include improved software quality, reduced deployment failures, and faster system-recovery times.

Making the switch to DevOps means that production problems “can either be proactively averted or identified before they can cause significant harm,” says Alan Zucker, a DevOps consultant in financial services and founding principal of Project Management Essentials.

One way to achieve that through DevOps is “chaos monkey testing”—an engineering practice first developed by Netflix a decade ago—in which developers introduce and test unexpected errors and other failures in a safe production environment. That allows ops teams to learn how to quickly respond and address problems.

At a large bank where Zucker led a two-year DevOps implementation program, the number of annual software releases deployed into production jumped from about 250 to more than 1,000, he says. The number of critical production issues fell to zero. One app that automated the release process, he adds, slashed deploy times from four days to four hours

Another tenet of DevOps is using automating manual processes. That doesn’t just reduce deploy times and lower cost. It also helps developers be more productive on tasks that matter. Automation “eliminates the non-value-add work, such as emails and conference calls to discuss deployments, service requests, and change records,” says Jamil Mina, chief solutions architect for financial services at Red Hat.

DevOps also eases the burden on testing for operations teams. Instead of handing off testing as a final step, engineering and operations teams jointly follow a “pipeline” process that ensures every change in code is checked against a bank’s software stack. With the help of automation tools, “it sets a standard for how work is done and removes the potential for human error,” says Mina.

Rocky road to adoption

For banks and insurance companies that have worked with a mixed bag of legacy technologies for years, switching to a new, unified development process like DevOps isn’t easy. “Software engineering isn’t nearly as easy in a bank as it is with a large software company,” says Mina. “Banks have relied so heavily on legacy applications that they don’t have the time or budget to rewrite.”

Devops brings new cultural challenges as well. Collaboration between dev and ops encourages everyone to become an equal stakeholder in new processes and protocols. “The pipeline should be designed to meet their needs, to give them flexibility with the build pipeline, and visibility into deployment,” Mina adds.

Even for smaller players like OnlineLoans, the switch to DevOps has had a unifying impact. Anyone involved in a software sprint also attends a daily meeting with developers. The new approach “prevents cross-team stakeholders from having competing interests,” Gendron says.

That kind of kumbaya spirit is a new one for many financial services CIOs, but one they’ll likely need more of in the years ahead.