This MSc thesis project focuses on the design and implementation of a proof‑of‑concept framework for generating, visualising, and comparing explainable AI (XAI) outputs across fundamentally different model architectures, specifically classical machine‑learning models and large language models (LLMs).

As AI systems are increasingly deployed in regulated domains such as healthcare, developers are often required to justify and compare model behaviour across multiple candidate approaches. While a wide range of XAI methods and visualisation tools exist, current frameworks typically support only a narrow class of models, most commonly scikit‑learn‑based pipelines. There is no widely used open‑source framework that enables consistent comparison of explanations across classical ML models and LLMs within a single environment.

The central aim of this project is to address that gap by developing a modular, extensible framework that allows explanations from different model types to be generated, stored, and explored side‑by‑side, while making clear the limits of comparability between explanation paradigms.

The framework is being designed around the following principles:

• support for multiple model architectures, from classical scikit‑learn models to LLM‑based pipelines
• integration of appropriate XAI methods for each model type (e.g. post‑hoc feature attribution for classical ML, intrinsic reasoning outputs for LLMs)
• consistent and interpretable visualisations that avoid misleading cross‑model comparisons
• modular extensibility using an adapter‑based design to support future models and explanation techniques

The technical implementation is planned as a Python‑based framework, with automated pipelines for running XAI methods and standardising their outputs, combined with a lightweight interactive dashboard (e.g. Streamlit or Dash) for exploring and comparing explanations. Computationally expensive explanation methods, particularly for LLMs, are designed to be run offline or on suitable hardware, with results imported into the dashboard for interactive analysis.

To ground the comparison in a realistic application context, the framework will be demonstrated using a healthcare‑motivated text classification task based on synthetic clinical conversational data. This allows the project to explore explainability in a domain where transparency, traceability, and trust are critical, while avoiding ethical and privacy constraints associated with real patient data.

Evaluation of the framework will combine technical validation (correctness, robustness, extensibility) with human‑centred assessment, focusing on whether the generated explanations are perceived as clear, trustworthy, and useful by representative users. Rather than treating accuracy as the primary outcome, the project emphasises the comparability, clarity, and interpretive value of explanations across model types.

At its current stage, the project has completed background research, requirements analysis, and architectural design. Implementation and evaluation are ongoing. The final outcome will be a working proof‑of‑concept framework, an interactive dashboard, and a detailed evaluation of how different explanation paradigms behave when applied to the same task.