Understanding Industrial Log Analysis: A Multi-Dataset Evaluation of Parsing and Anomaly Detection.
Abstract
Log analysis plays a critical role in monitoring and maintaining the safety of industrial software systems. However, most existing research relies heavily on benchmark datasets derived from legacy or open-source systems, which fail to capture the structural diversity and operational complexity of realworld industrial logs. In this study, we present a comprehensive empirical evaluation of log parsing and anomaly detection models across four diverse datasets, including three collected from largescale industrial software deployed in manufacturing, process control, and energy monitoring environments. Our analysis reveals that state-of-the-art models-particularly rule-based parsers and supervised detectors-experience substantial performance degradation when applied to industrial settings. To address this gap, we introduce a unified evaluation framework using representative training subsets, and we highlight the effectiveness of semisupervised and LLM-based approaches in handling heterogeneous, low-resource log environments. The findings offer practical insights into the limitations of current log analysis techniques and suggest design principles for building more robust, domainadaptive solutions for industrial software risk mitigation. Index Terms-Industrial Software, Log Parsing, Anomaly Detection, Semi-Supervised Learning, Large Language Models I. I NTRODUCTION Log data has become an essential asset in maintaining the safety, stability, and reliability of industrial software systems [1]-[3]. Generated automatically during system execution, logs provide valuable records of internal operations, offering insights into system behaviors, failures, and potential risks [4]- [6]. As software continues to increase in complexity and scale, the role of automated log analysis-especially log anomaly detection-has become increasingly critical for timely risk mitigation and operational resilience [7]-[9]. A significant body of research on log anomaly detection has relied on publicly available datasets, most notably those hosted on LogHub [10]. While these datasets have served as important benchmarks, they exhibit several limitations [11]. One significant limitation of LogHub is