Evaluating the Performance of a Fake News Model on A Domain-Specific and Heterogeneous Dataset to Improve Detection

Abstract

The rapid evolution of technology and the digital age has led to an increase in the spread of fake news, severely undermining the accuracy of information. This study aims to improve fake news detection methods in distinct domains through in-depth dataset analysis using a Convolutional Neural Network (CNN), the research-trained models using an optimized CNN model on publicly available datasets. The findings show that machine learning models trained on domain-specific datasets can accurately identify the nuances of fake news unique to those domains. Compared to models trained on broader datasets, the results demonstrate that models trained on domain-specific data achieved higher accuracy, precision, recall, and F1-score, increasing from 68% to 99% across all metrics when compared with a baseline CNN model. However, while domain-specific models perform exceptionally well in their respective contexts, models trained on a diverse range of datasets exhibit greater generalizability across domains. These findings suggest that dynamic and robust fake news detection systems should integrate both heterogeneous datasets and domain-specific features to enhance effectiveness.