Towards Lightweight LLM Software Solutions for InsurTech: A Framework for Scalable Question Answering Systems.
Abstract
The integration of Large Language Models (LLMs) into software systems is transforming regulated sectors like insurance, where precision, compliance, and efficiency are essential. While proprietary LLMs like GPT-4 offer state-of-the-art performance, their closed-source nature and high computational demands constrain adoption in privacy-sensitive and cost-restricted InsurTech environments. In response, this paper investigates how lightweight, open-source LLMs can be effectively deployed for domain-specific question answering in insurance, emphasizing software engineering considerations such as modularity, inference stability, and prompt orchestration. We propose a softwareengineered evaluation framework tailored to insurance-related tasks, featuring modular prompt management, automated rubricbased evaluation, and backend support for reproducibility and compliance tracking. A curated benchmark dataset derived from the Hong Kong Insurance Intermediaries Qualifying Examination (IIQE) is constructed to reflect real-world regulatory and operational challenges. Ten open-source models are systematically evaluated across four question types using both standard and Chain-of-Thought (CoT) prompting strategies. Our findings show that compact models such as DeepSeek-R1-1.5B achieve strong accuracy with minimal resource consumption, making them suitable for practical deployment. CoT prompting further enhances reasoning performance, particularly for models with 3B parameters or more. With proper prompt design and modular deployment, lightweight LLMs can support secure, efficient, and interpretable InsurTech applications, enabling trustworthy AIdriven software systems in regulated domains. Index Terms-Large Language Models, Prompt Engineering, InsurTech, Chain-of-Thought Reasoning, Software System Integration I. I NTRODUCTION The rise of Large Language Models (LLMs) such as GPT-4, LLaMA, and DeepSeek has significantly advanced the capabilities of natural language processing, driving progress in question answering, code generation, and conversational AI. As summarized by Hou et al. [1], LLMs are rapidly transforming various stages of the software development lifecycle, includin