Vysvětlování výstupu modelů zpracování přirozeného jazyka pro úlohu ověřování faktů

Abstract

This thesis explores interpretability methods and the possibilities of their application to natural language processing (NLP) models used within a fact-checking pipeline. More specifically, it focuses on the application of two local, model-agnostic interpretability methods LIME and SHAP to natural language inference (NLI) models used to infer a veracity label from a claim and a context. In this work, we modify and apply SHAP and LIME interpretability methods to the NLI models and develop a text-augmented version for LIME. Later, we test various parameter settings to find the optimal parametrization for each method which we then compare in a binary forced-choice experiment with human-grounded evaluation. For both datasets used within the the project, SHAP is evaluated to produce more helpful explanations.

This thesis explores interpretability methods and the possibilities of their application to natural language processing (NLP) models used within a fact-checking pipeline. More specifically, it focuses on the application of two local, model-agnostic interpretability methods LIME and SHAP to natural language inference (NLI) models used to infer a veracity label from a claim and a context. In this work, we modify and apply SHAP and LIME interpretability methods to the NLI models and develop a text-augmented version for LIME. Later, we test various parameter settings to find the optimal parametrization for each method which we then compare in a binary forced-choice experiment with human-grounded evaluation. For both datasets used within the the project, SHAP is evaluated to produce more helpful explanations.