Cohérence logique dans les grands modèles de langue

Keywords

LLM, NLP, logic, semantic, reasoning, sustainability

Subject details

Recent years have witnessed rapid advances in natural language processing, with large language models (LLMs) achieving state-of-the-art performance across many tasks and being increasingly deployed in safety-critical applications. Despite their fluency and generalization capabilities, LLMs exhibit systematic failures inconsistency, raising concerns about their reliability, robustness, and susceptibility to misuse or unintended behaviour in high-stakes settings [Jang et al., 2022a,b; Liu et al., 2025; Novikova et al., 2025; Cheng et al., 2025]. Current evaluation paradigms largely emphasize task accuracy [Liu et al., 2025], providing limited insight into whether model behaviour is internally coherent or resilient to perturbations and attacks. This PhD project investigates to what extent formal semantic representations can be used to evaluate, monitor, and improve LLM consistency, with a focus on logical consistency as a component of robust and secure model behaviour. By leveraging tools from formal semantics and logic, the project seeks to develop principled methods for detecting inconsistency-driven vulnerabilities and enabling verifiable and debuggable LLM systems [Toroghi et al., 2024]. The long-term objective is to contribute to the design of LLM-based systems whose behaviour can be formally characterized and constrained, thereby improving their safety and trustworthiness across languages and deployment settings. LLM consistency is defined by Jang et al. [2022a] as a model's ability to make coherent, non-contradictory decisions. They distinguish semantic, logical, and factual consistency, with logical consistency including negational, symmetric, transitive, and additive properties. While semantic and factual consistency have been widely studied, logical consistency remains comparatively underexplored from a formal perspective. Existing evaluation methods mainly rely on benchmark-based logical attacks [Nakamura et al., 2023] or on verifying intermediate reasoning artifacts translated into logical forms and checked with external solvers [Cheng et al., 2025]. These approaches present several limitations: they are task-specific and heuristic, provide no formal guarantees beyond evaluated benchmarks, and mainly address short text segments. Long-text and discourse-level reasoning therefore remain largely understudied [Jang and Lukasiewicz, 2023], despite the fact that many real-world failures arise from document-level inconsistencies. Moreover, current work focuses almost exclusively on English, leaving open questions about typological diversity and low-resource settings. This project addresses these limitations through formal semantic and symbolic approaches to monitoring, evaluation, and mitigation. A central objective is to exploit the YARN semantic formalism [Pavlova et al, 2025] as a structured intermediate representation of meaning and to investigate its systematic transformation into logical representations suitable for formal verification. Research directions include: (i) defining explicit logical constraints to detect inconsistencies in model outputs, including in long-text settings; (ii) exploring symbolic verification and repair strategies, such as constrained decoding; and (iii) investigating richer logical formalisms and multilingual extensions to assess robustness across languages and under uncertainty. This project adopts the view that formal semantic approaches support precise and compact meaning representations that can be implemented in smaller, specialized models rather than relying on large-scale architectures. Accordingly, part of the research will focus on designing AI systems with high semantic quality and logical reliability while limiting dependence on extreme model scaling. Environmental impact will be considered as an evaluation criterion alongside accuracy and robustness, taking into account computational cost and energy consumption in line with sustainable and green AI principles.