Enhancing Rumor Veracity Classification Beyond Traditional Methods

Towards improving Rumor Veracity Classification and beyond

Introduction and Feature Extraction Revision and Improvement of Model Increase Model Interpretability based on Attention Mechanism and Future work

Rumor Detection and Some Subtasks? Veracity classification Verify the truth of the stated events We sometimes define the rumor as the false information or fake news (labeled false in this tasks) Stance detection Test comment s stance to the source tweets Consider simple stance, such as support, deny, confuse, comment.

Previous Method Deep learning for Rumor Detection Detecting Rumors from Microblogs with Recurrent Neural Networks Motivation: Using word2vec method to transfer tweets into words vector. Apply RNN model to give a general prediction of the veracity All-in-one: Multi-task Learning for Rumor Verification Motivation: By using the connections between different rumor subtasks and using the multi-task learning mechanism to increase the amount of information during learning, the performance of difficult subtasks can be improved

Feature For Rumor Detection Feature Used in Deep Learning Only Word embedding Feature Used in Traditional Method Content Feature Length, has negative or positive words. User credibility Feature Whether robots? Register age, description, followers. Social network structure Feature Source, tweets and retweets.

Feature For Rumor Detection Our Method Traditional Feature+ Word Embedding Word Embedding Word2vec method Tweet Feature Words number Has picture, URL Is Source tweet User Info Feature (credibility) Is verified Friends number Followers number

Experiment and Result with Additional Feature Method Method Veracity Acc Veracity Acc Veracity Marco f1 Veracity Marco f1 Stance acc Stance acc stance stance marco marco RNN 0.510 0.514 \ \ MTL2 0.556 0.554 0.617 0.232 MTL2 with Additional feature 0.592 0.556 0.574 0.229 Result: better veracity classification score, but lower stance classification score

Model reimplementation & revision

Model Input use source tweet and its reply tweet construct several branches each branch as an input sequence

Multi-task Learning Framework Sub-task including: Rumor stance classification Rumor veracity classification Rumor/Non-rumor detection Main thought: Subtasks may have some common features Use a shared module to perform feature extraction A typical Multi-task learning model for rumor resolution

Model Revision All-in-one (COLING, 2018) MTL model Rumor Detection by Exploiting User Credibility Information, Attention and Multi-task Learning(2019, ACL)

Model Revision Use different feature + content for different sub-task More Information injected into Stance/Rumor Detection Layer Apply Attention Module: each hidden state matters Workflow Comparison: sub sub- -task A task A MTL2 tweet tweet- -embedding embedding User feature User feature Content feature Content feature shared layer shared layer Sub Sub- -task B task B Revision Step 1 hidden output hidden output shared layer shared layer All feature embedding All feature embedding Step 2 tweet tweet- -embedding embedding User feature User feature sub sub- -task A task A hidden output hidden output hidden output hidden output tweet tweet- -embedding embedding Content feature Content feature Sub Sub- -task B task B

Model Revision Result Comparison: Method Method Veracity Acc Veracity Acc Veracity Macro F1 Veracity Macro F1 Stance Acc Stance Acc Stance Macro F1 Stance Macro F1 MTL with additional feature 0.592 0.556 0.574 0.229 Revision model 0.629 0.629 0.607 0.607 0.691 0.232 Revision model +attention 0.629 0.629 0.607 0.607 0.711 0.711 0.244 0.244 More about attention. . . More about attention. . .

Attention Mechanism

Pros of Attention Mechanism Dynamic weight, which can help capture important information. Bring more interpretability to the model. High performance and efficiency.

Attention Mechanism in the model ?= [ ??, ??] ??= ??tanh(? ?+ ?) ? = ???????(?) ? = ?? ? Here ?? is the attention of a specific tweet.

Example about Interpretability @MichaelSkolnik: Every 28 hours a black male is killed in the United States by police or vigilantes. #Ferguson @Dustin_Jeffrey @MichaelSkolnik you can google it. Same as whites and blacks smoke weed at some rate but blacks 8 more times 2 b arrested @MadChad412 @MichaelSkolnik and as far as Google goes. I'm published on google being biased as all hell. I don't trust everything I google @MadChad412 @MichaelSkolnik that's fine, but I would like definitive proof before making a hypothesis Every 28 hours a black male is killed in the United States by police or vigilantes. #Ferguson @Dustin_Jeffrey @MichaelSkolnik he is a legit source. Can I have your source? 0.43 0.004 0.11 Attention weight 0.0002 0.46 Some replies may contribute more to the predictions about the veracity, such as evidence and question.

Future workWord-level Attention in Stance Detection Reasoning about Entailment with Neural Attention(ICLR,2016) This may help find the sensitive words about the stance, which can improve the feature extraction.

Future work: Word-level Attention in Feature Extracting Use word level attention to mine sensitive words with respect to the stance and veracity of the tweets. We can further use the sensitive vocabulary to do feature extraction.