ServiceNow Research attends EMNLP 2022

The 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP 2022) was held in Abu Dhabi, United Arab Emirates, Dec. 7-11, 2022.

ServiceNow Research had three papers accepted at EMNLP 2022:

• Azimuth: Systematic Error Analysis for Text Classification by Gabrielle Gauthier Melançon, Orlando Marquez, Lindsay Brin, Chris Tyler, Frédéric Branchaud-Charron, Joseph Marinier, Karine Grande, Di Le

• UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models by Tianbao Xie, Chen Wu, Peng Shi, Ruiqi Zhong, Torsten Scholak, Michihiro Yasunaga, Chien-Sheng Wu, Ming Zhong, Pengcheng Yin, Sida I. Wang, Victor Zhong, Bailin Wang, Chengzu Li, Connor Boyle, Ansong Ni, Ziyu Yao, Dragomir Radev, Caiming Xiong, Lingpeng Kong, Rui Zhang, Noah A. Smith, Luke Zettlemoyer, Tao Yu

• On the Compositional Generalization Gap of In-Context Learning by Dzmitry Bahdanau, Arian Hosseini, Aaron Courville, Alessandro Sordoni, Ankit Vani
  
  

Azimuth system demo

Orlando Marquez, an applied research scientist in the Advanced Technologies Group at ServiceNow, attended the event in person and presented the poster and system demo for Azimuth on Dec. 9. Azimuth is an open-source tool for error analysis in natural language processing (NLP) text classification. It is designed to help artificial intelligence (AI) practitioners find and address areas in which their models are not performing optimally by supplying a range of techniques for data set analysis and model quality assessment.

For data set analysis, Azimuth provides AI practitioners with tools for understanding the characteristics of the data set, including class distribution and any class imbalances that may exist. It also allows users to analyze the quality of the data, such as by identifying data shifts between data set splits and finding errors and inconsistencies.

For model quality assessment, Azimuth uses a range of techniques to help AI practitioners understand how the model is making decisions, recognize patterns in its errors, assess confidence in its predictions, and learn how it behaves under different conditions.

Azimuth has the potential to significantly improve the error analysis stage of the machine learning (ML) development cycle, making it more systematic and efficient. The reception to Azimuth was incredibly positive, as it’s a well-designed, open-source tool to help perform systematic error analysis of NLP models.

We received feedback regarding the need to support other tasks, such as summarization, and more languages, such as Arabic. We invite the AI community to contribute to the project by helping us add support for new tasks and languages.

Thank you and congratulations to the organizers and volunteers for hosting another momentous event, and to all the researchers who had their work accepted at the conference.

Find out more about ServiceNow Research.

© 2023 ServiceNow, Inc. All rights reserved. ServiceNow, the ServiceNow logo, Now, and other ServiceNow marks are trademarks and/or registered trademarks of ServiceNow, Inc. in the United States and/or other countries. Other company names, product names, and logos may be trademarks of the respective companies with which they are associated.