Chart2Code-MoLA: Efficient Multi-Modal Code Generation via Adaptive Expert Routing.
Abstract
Chart-to-code generation is a critical task in automated data visualization, translating complex chart structures into executable programs. While recent Multi-modal Large Language Models (MLLMs) improve chart representation, existing approaches still struggle to achieve cross-type generalization, memory efficiency, and modular design. To address these challenges, this paper proposes C2C-MoLA, a multimodal framework that synergizes Mixture of Experts (MoE) with Low-Rank Adaptation (LoRA). The MoE component uses a complexity-aware routing mechanism with domain-specialized experts and load-balanced sparse gating, dynamically allocating inputs based on learnable structural metrics like element count and chart complexity. LoRA enables parameter-efficient updates for resource-conscious tuning, further supported by a tailored training strategy that aligns routing stability with semantic accuracy. Experiments on Chart2Code-160k show that the proposed model improves generation accuracy by up to 17%, reduces peak GPU memory by 18%, and accelerates convergence by 20%, when compared to standard fine-tuning and LoRAonly baselines, particularly on complex charts. Ablation studies validate optimal designs, such as 8 experts and rank-8 LoRA, and confirm scalability for real-world multimodal code generation. Index Terms-Chart-to-Code Generation, Multi-Modal Learning, Mixture of Experts (MoE), Low-Rank Adaptation (LoRA) I. I NTRODUCTION With the growing reliance on data visualization for insight communication, automated understanding of visual content has become central to intelligent systems [1]. Chart-to-code generation refers to the task of converting visualizations such as bar, line, and scatter plots into executable programs, which requires precise interpretation of visual semantics and syntactically correct code generation [2]. Early methods used rule-based extraction from synthetic images or well-structured images [3] [4], leveraging predefined heuristics to identify chart elements. Subsequently, neural approaches introduced encoder-decoder architectures to tackle chart-to-data tasks [5] [6], learning to map pixel-level inputs to structured representations. These advance