Faculty of Science, Technology and Medicine
Faculty of Law, Economics and Finance
Faculty of Humanities, Education and Social Sciences
Interdisciplinary Centre for Security, Reliability and Trust
Luxembourg Centre for Systems Biomedicine
Luxembourg Centre for Contemporary and Digital History
Luxembourg Centre for European Law
Luxembourg Centre for Socio-Environmental Systems
The Doctoral School in Science and Engineering is happy to invite you to Yewei SONG’s defence entitled
Intelligent Conversation System for Multilingual Financial Industry
Supervisor: Prof Jacques KLEIN
The financial services industry is undergoing a paradigm shift towards automated, intelligent customer interaction. While Large Language Models (LLMs) offer promising capabilities for natural language understanding, their deployment in the regulated banking sector is hindered by at least three barriers:
(1) the lack of support for low-resource languages (LRLs) such as Luxembourgish;
(2) the inability to interact reliably and securely with subsystems like SQL databases or via Application Programming Interface (API) function calling under strict privacy constraints;
(3) the inherent stochasticity of generative outputs, which threatens operational stability.
This dissertation proposes a holistic architecture for an Intelligent Conversation System tailored to the multilingual financial industry, addressing these challenges through a tripartite framework.
The first part addresses the linguistic complexities of the Luxembourgish financial environment. We challenge the prevailing “scaling laws” of LLMs by demonstrating that Small Language Models (SLMs), when augmented with high-quality monolingual data and knowledge distillation from high-capacity teacher models, can achieve translation parity with significantly larger models. Our research validates that targeted fine-tuning on LRLs does not induce catastrophic forgetting, thereby establishing a resource-efficient pathway for inclusive, multilingual banking agents.
The second part bridges the gap between natural language intent and formal banking protocols. In the domain of Text-to-SQL, we tackle the conflict between evaluation rigor and data privacy. We introduce two novel, non-execution-based metrics, Tree Similarity of Editing Distance (TSED) and the SQL Query Analysis Metric (SQAM), which allow for the static, structural validation of generated queries without exposing sensitive production data. In the domain of API interaction, we present CallNavi, a comprehensive benchmark for evaluating agentic routing. We identify critical limitations in current models regarding nested function calls and propose “backward inference” strategies that significantly enhance the reliability of structured output generation.
The third part focuses on the quality assurance of the system’s code generation capabilities. Recognizing that functional correctness (Pass@k) is insufficient for safety-critical operations, we introduce a framework for quantifying Output Stability. We propose entropy-based metrics, specifically Structural Cross-Entropy (SCE) and Jensen-Shannon Divergence (JSD) applied to Abstract Syntax Trees (ASTs). These metrics provide a language-agnostic method to measure the structural consistency of the model’s reasoning process, ensuring that the system is not merely correct by chance, but robust and reproducible.
Collectively, this work contributes a validated blueprint for deploying sovereign, privacy-preserving, and stable AI agents, moving the state of the art from general-purpose chatbots to specialized, trustworthy financial assistants.