Misinformation mitigation using automated fact-verification methods

The proliferation of false and misleading information, fuelled by the rapid progress in artificial intelligence (AI), poses a significant societal threat, as highlighted in the World Economic Forum’s report in 2024. To address this important challenge, we developed methods for automated fact verification that use an existing knowledge source in order to find evidence for supporting a claim, refuting a claim, or deciding that there is not sufficient evidence to make a decision. To this end, we developed two approaches:

1. Automated Fact-Verification using Google search results as the knowledge source: For the AVeriTeC public fact verification challenge, we developed a sophisticated multi-stage pipeline that integrates cutting-edge techniques for evidence retrieval and question generation. Our system leverages cross-encoders and large language models (LLMs) to enhance accuracy across key tasks, including evidence extraction, question generation, and veracity prediction. By refining these processes, our approach significantly outperforms baseline models, particularly in handling complex claims that require nuanced reasoning. Through extensive experiments and ablation studies, we provide valuable insights into the strengths and limitations of our method, underscoring the critical role of evidence sufficiency and context dependency in automated fact-checking systems.
2. Automated Fact-Verification using Wikipedia as the knowledge source: We introduce a new approach to improve fact verification by refining how evidence is retrieved and ranked. Our Multi-stage ReRanking (M-ReRank) method overcomes the limitations of single-stage evidence extraction by using advanced ranking techniques. Tested on the FEVEROUS dataset, our system significantly boosts accuracy compared to the state of the art, achieving a 93.63% recall rate for Wikipedia pages. It also improves sentence, table, and cell retrieval, outperforming previous models with recall gains of 7.85%, 8.29%, and 3%, respectively.

More details in the related outputs

Shrikant Malviya and Stamos Katsigiannis. 2024. Evidence Retrieval for Fact Verification using Multi-stage Reranking. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 7295–7308, Miami, Florida, USA. https://doi.org/10.18653/v1/2024.findings-emnlp.428

Shrikant Malviya and Stamos Katsigiannis. 2024. SK_DU Team: Cross-Encoder based Evidence Retrieval and Question Generation with Improved Prompt for the AVeriTeC Shared Task. In Proceedings of the Seventh Fact Extraction and VERification Workshop (FEVER), EMNLP 2024, pages 99–107, Miami, Florida, USA. https://doi.org/10.18653/v1/2024.fever-1.11