Detecting the doubt effect and subjective beliefs using neural networks and observers’ pupillary responses

Xuanying Zhu, Zhenyue Qin, Tom Gedeon, Richard Jones, Md. Zakir Hossain, Sabrina Caldwell

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

We investigated the physiological underpinnings to detect the ‘doubt effect’ – where a presenter’s subjective belief in some information has been manipulated. We constructed stimulus videos in which presenters delivered information that in some cases they were led to doubt, but asked to “present anyway”. We then showed these stimuli to observers and measured their physiological signals (pupillary responses). Neural networks trained with two statistical features reached a higher accuracy in differentiating the doubt/ manipulated-belief compared to the observers’ own veracity judgments, which is overall at chance level. We also trained confirmatory neural networks for the predictability of specific stimuli and extracted significant information on those stimulus presenters. We further showed that a semi-unsupervised training regime can use subjective class labels to achieve similar results to using the ground truth labels, opening the door to much wider applicability of these techniques as expensive ground truth labels (provenance) of stimuli data can be replaced by crowd source evaluations (subjective labels). Overall, we showed that neural networks can be used on subjective data, which includes observer perceptions of the doubt felt by the presenters of information. Our ability to detect this doubt effect is due to our observers’ underlying emotional reactions to what they see, reflected in their physiological signals, and learnt by our neural networks. This kind of technology using physiological signals collected in real time from observers could be used to reflect audience distrust, and perhaps could lead to increased truthfulness in statements presented via the Media.

Original languageEnglish
Title of host publicationNeural Information Processing - 25th International Conference, ICONIP 2018, Proceedings
EditorsSeiichi Ozawa, Andrew Chi Sing Leung, Long Cheng
PublisherSpringer-Verlag
Pages610-621
Number of pages12
ISBN (Print)9783030042110
DOIs
Publication statusPublished - Jan 1 2018
Event25th International Conference on Neural Information Processing, ICONIP 2018 - Siem Reap, Cambodia
Duration: Dec 13 2018Dec 16 2018

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11304 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other25th International Conference on Neural Information Processing, ICONIP 2018
CountryCambodia
CitySiem Reap
Period12/13/1812/16/18

    Fingerprint

Keywords

  • Doubt
  • Information veracity
  • Neural networks
  • Pupillary responses
  • Semi-unsupervised training
  • Subjective belief
  • Trust

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Zhu, X., Qin, Z., Gedeon, T., Jones, R., Hossain, M. Z., & Caldwell, S. (2018). Detecting the doubt effect and subjective beliefs using neural networks and observers’ pupillary responses. In S. Ozawa, A. C. S. Leung, & L. Cheng (Eds.), Neural Information Processing - 25th International Conference, ICONIP 2018, Proceedings (pp. 610-621). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11304 LNCS). Springer-Verlag. https://doi.org/10.1007/978-3-030-04212-7_54