Journal Article

When parsing goes wrong: Revisiting log anomaly detection from parsing to deployment

Yicheng Sun, Jacky Wai Keung, Yishu Li, Xiaoxue Ma, Hi Kuen Yu
Journal of Systems and Software 2027 10.1016/j.jss.2026.113161

Abstract

Context: Log analysis is critical for the reliability and security of modern software systems. Although log parsing and anomaly detection are tightly connected in practice, prior studies have largely evaluated them in isolation, and the role of public data in supporting industrial anomaly detection remains unclear. Object. This study investigates how log parsing errors affect downstream anomaly detection, how external public logs influence industrial anomaly detection under data scarcity, and how practitioners perceive these issues in real-world systems. Method: We conduct a unified empirical study over 16 public LogHub datasets and four proprietary industrial datasets. We analyze the outputs of multiple rule-based and LLM-based parsers, identify six common log parsing error types through expert review, and evaluate their downstream impact using five anomaly detection models. We further assess cross-dataset data augmentation on industrial datasets and complement the quantitative analysis with interviews with 11 engineers involved in the development and maintenance of the industrial systems. Results: Log parsing errors propagate systematically to anomaly detection, and correcting such errors leads to statistically reliable overall improvements, especially on industrial datasets. Cross-dataset augmentation can improve anomaly detection under data scarcity, but its benefits are highly model dependent and may lead to negative transfer when domain compatibility is limited. Practitioner feedback confirms that industrial logs are more heterogeneous, context-dependent, and operationally constrained than public benchmarks, which helps explain the stronger impact of parsing errors and the mixed transfer results. Conclusion: Our findings show that reliable industrial log analysis requires jointly considering parsing fidelity, data compatibility, model robustness, and practitioner trust. More broadly, the study highlights the need to treat log parsing, data curation, and anomaly detection as a tightly coupled system design problem.

Keywords