Bringing basketball to a global audience
The National Basketball Association (NBA) is the world’s premier professional basketball organization, attracting a massive global following. The NBA generated a record 25.2 billion video views across the league’s global social media channels and the NBA App during the 2023-24 season.
The pace, precision, and physicality of the game is breathtaking, with its greatest players transcending the court, becoming sports and cultural icons. While the NBA’s teams are in North America, the appeal of the league is universal, with many millions of fans throughout Europe, Asia, and South America.
The NBA is also one of the most innovative sports organizations, loved by fans in 214 countries. It invests in cutting edge technologies, including public cloud, containerized applications, modern software development pipelines, and AI to create uniquely immersive, interactive, and engaging experiences.
Some prominent examples of this applied technology are NBA TV, the NBA League Pass live-game streaming service, and the NBA App, which cater to the insatiable worldwide appetite among fans for news and highlights, as well as access to a vast archive of live game coverage, video, data, and player statistics. These services are every fan’s virtual courtside ticket, enabling subscribers to enjoy an enhanced live and on-demand game experience on multiple devices, available in multiple languages, anytime, anywhere.
To ensure that its content, game coverage and other digital services deliver a seamless, world-class experience for fans, the NBA employs cutting-edge technology. Since 2020, ServiceNow has played an important role in ensuring the quality, experience, and uptime of the NBA’s applications and infrastructure.
Enabling creativity to flourish
One of the keys to the NBA’s enduring success in capturing the imagination of fans worldwide is its philosophy of “meeting our teams where, and how they work.” Developers working on its pipeline of new features, enhancements, and innovations are empowered to use the tools and technologies best suited for their creative needs, as well as the requirements of their applications.
This best-in-class approach also applies to the tools they deploy to optimize application and service performance and availability. “Our teams have the authority to use whatever tooling they need to do a great job and manage their applications, including using their own monitoring tools,” explains Mark Bennett, the NBA’s SVP, Head of Technology & Infrastructure Operations.
ServiceNow has seamlessly integrated into this environment, providing AIOps and case management capabilities realized in the native environments developers have chosen. For example, critical information surfaced in ServiceNow is proactively shared to developers directly within their team communication environment, rather than requiring developers to ‘swivel chair’ into ServiceNow itself. This streamlines and simplifies DevOps, with ServiceNow acting as an ‘invisible guiding hand’ to keep operations on-track.
“Our Site Reliability Engineering (SRE) team supports developers with an enterprise-wide perspective. We ensure frictionless communication between teams, sharing of information and best practices, and we create processes behind the scenes that empower our developers, without getting in their way,” continues Mark.
“An important dimension of the support we provide for our teams comes with our use of ServiceNow Health Log Analytics (HLA) and IT Operations Management technology to deliver AIOps capabilities.”
Live broadcasting, whether over the air or streaming on the Internet, is an unforgiving environment, with viewers instantly aware of any reduction in video quality, lags, hangs, or session drops. The NBA has decades of operational experience bringing a world-class product to its subscribers and is constantly monitoring diverse systems and components that must work perfectly in combination. Extensive use of public cloud resources and Kubernetes’ elastic auto-scaling capabilities handles the constantly changing demand for streaming services.
“While we’re very experienced and have a lot of operational intelligence in that environment, ServiceNow is helping us to constantly mature our capabilities and take our game to the next level,” says Mark.
Striving for perfection
ServiceNow’s AIOps sits across the NBA’s entire ecosystem, integrated with all its bespoke monitoring tools, providing a crucial additional layer of intelligence by analyzing and correlating insights across different services and data sources. AIOps goes further and digs deeper than is humanly possible, unearthing what Mark describes as “the needles in the haystack.”
“Originally our goal was to solve problems faster, but ServiceNow has delivered much more. Its machine-learning capabilities and predictive analytics enable us to identify potential issues at the source and fix them before they go into production. We’re now pre-emptively identifying issues, rather than resolving problems down the line when they negatively affect the user experience.”
One of those needles unearthed by ServiceNow provides a great example of the NBA’s constant drive to perfect the user experience. Viewing classic games from the NBA archive is a popular pastime for many basketball fans. HLA discovered a glitch in the content retrieval system, resulting in games occasionally failing to load on customers’ devices. “This is what I love about ServiceNow,” says Mark. “It reinforces us and provides the grit that we need. This wasn’t a problem we were aware of or were looking for. But every interaction counts in the fan’s experience. ServiceNow keeps helping us to raise the bar, to be as good as we can possibly be.”
Identifying the root cause
HLA is a key component of the ServiceNow AIOps solution. By collecting, correlating, and understanding machine-generated log data in real-time, ServiceNow AIOps can predict technology issues before they affect end users. HLA discovers deviations and sends alerts about possible issues. It also identifies and triages anomalies using machine learning, and groups the anomalies together to identify root causes. In addition, HLA doesn’t require users to set threshold parameters as it detects anomalies users didn’t know were occurring.
The scale—and impact—of ServiceNow AIOps on the NBA’s monitoring operation is enormous. HLA analyzes more than 200 million logs generated by the NBA App each day, a number that is constantly rising. Through correlation and analysis of the data, HLA produces between 20 and 30 alerts requiring investigation, with two or three resulting in remedial action.
“That’s a 99.999% reduction in noise, and those two or three items requiring additional investigation are always important, things we can take action on right away,” says Site Reliability Engineer, Jamie Phillips.
Engineers can improve results over time by providing feedback to the software about whether the alerts it surfaced were significant. HLA learns from this feedback and gets better at ignoring the unimportant and focusing on the critical.
“Teams are looking at their logs in real-time typically, without machine learning capability,” adds Mark. “HLA helps us to break through the noise and the traffic that humans can’t process within the huge volume of logs. It ensures that we’re monitoring additional things not in view, aggregates those logs and surfaces issues that are most important, providing a level of intelligence on top of human eyes.”
When an issue is identified, the team launches an incident in ServiceNow IT Service Management (ITSM), bringing all stakeholders together via integration with a team collaboration tool, with the resolution and learning points captured in ServiceNow and shared across the organization.
Says Mark: “By integrating ServiceNow AIOps, we have strengthened our incident management across the business. We’re now taking what we have learned from our consumer apps and applying those lessons to our back office and other apps, so we get the benefit of that maturity.”
Delivering trust and confidence
As the SRE team instructs ServiceNow AIOps to differentiate between issues, focusing on those with most significance, “the result is effective event suppression and great feedback for developers to make their lives easier and focus on enhancing their applications,” says Mark. “ServiceNow AIOps has eliminated incidents and we’re saving time because we’re no longer looking for the needle in the haystack. It’s like the hay has been taken away and only the needle remains.”
In addition to 90 predominantly direct-to-consumer services, now the NBA’s back office and network functions will also benefit from ServiceNow AIOps’ granular intelligence gathering, correlation, and analysis.
“Adoption of HLA has increased rapidly as our SRE teams see the tremendous value that it can offer,” says Jamie. “They appreciate that it’s been integrated into the tools they prefer to use and that it’s a frictionless, holistic experience for them. This results in driving great outcomes at speed. People are constantly asking for demos and want to funnel more and more data through HLA. It’s created a lot of trust and confidence amongst our teams.”
“ServiceNow sits in the middle of our ecosystem,” concludes Mark. “The ServiceNow AIOps product team has been one of the best that I have worked with. They’ve helped us figure out many of the nuances that we’ve encountered during a continuous learning process. It’s an excellent collaboration, we know that ServiceNow is with us all the way.”