ServiceNow recherche

BigCharts-R1: Enhanced Chart Reasoning with Visual Reinforcement Finetuning

Résumé

Charts are essential to data analysis, transforming raw data into clear visual representations that support human decision-making. Although current vision-language models (VLMs) have made significant progress, they continue to struggle with chart comprehension due to training on datasets that lack diversity and real-world authenticity, or on automatically extracted underlying data tables of charts, which can contain numerous estimation errors. Furthermore, existing models only rely on supervised fine-tuning using these low-quality datasets, severely limiting their effectiveness. To address these issues, we first propose BigCharts, a dataset creation pipeline that generates visually diverse chart images by conditioning the rendering process on real-world charts sourced from multiple online platforms. Unlike purely synthetic datasets, BigCharts incorporates real-world data, ensuring authenticity and visual diversity, while still retaining accurate underlying data due to our proposed replotting process. Additionally, we introduce a comprehensive training framework that integrates supervised fine-tuning with Group Relative Policy Optimization (GRPO)-based reinforcement learning. By introducing novel reward signals specifically designed for chart reasoning, our approach enhances model robustness and generalization across diverse chart styles and domains, resulting in a state-of-the-art chart reasoning model, BigCharts-R1. Extensive experiments demonstrate that our models surpass existing methods on multiple chart question-answering benchmarks compared to even larger open-source and closed-source models.

Publication
Conference on Language Modeling (COLM)
Ahmed Masry
Ahmed Masry
Visiting Researcher

Visiting Researcher at AI Frontier Research located at Toronto, ON, Canada.

Abhay Puri
Abhay Puri
Applied Research Scientist

Applied Research Scientist at AI Research Deployment​ located at Montreal, QC, Canada.

Juan A. Rodriguez
Juan A. Rodriguez
Visiting Researcher

Visiting Researcher at AI Frontier Research located at Montreal, QC, Canada.

Alexandre Piche
Alexandre Piche
Research Scientist

Research Scientist at AI Frontier Research located at Montreal, QC, Canada.

Christopher Pal
Christopher Pal
Distinguished Scientist

Distinguished Scientist at AI Research Partnerships & Ecosystem​ located at Montreal, QC, Canada.

David Vazquez
David Vazquez
Director of AI Research

Director of AI Research at AI Research Management located at Montreal, QC, Canada.

Perouz Taslakian
Perouz Taslakian
Research Lead

Research Lead at AI Frontier Research located at Montreal, QC, Canada.

Spandana Gella
Spandana Gella
Research Manager

Research Manager at AI Frontier Research located at Montreal, QC, Canada.