Detecting the doubt effect and subjective beliefs using neural networks and observers’ pupillary responses

Xuanying Zhu, Zhenyue Qin, Tom Gedeon, Richard Jones, Md. Zakir Hossain, Sabrina Caldwell

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We investigated the physiological underpinnings to detect the ‘doubt effect’ – where a presenter’s subjective belief in some information has been manipulated. We constructed stimulus videos in which presenters delivered information that in some cases they were led to doubt, but asked to “present anyway”. We then showed these stimuli to observers and measured their physiological signals (pupillary responses). Neural networks trained with two statistical features reached a higher accuracy in differentiating the doubt/ manipulated-belief compared to the observers’ own veracity judgments, which is overall at chance level. We also trained confirmatory neural networks for the predictability of specific stimuli and extracted significant information on those stimulus presenters. We further showed that a semi-unsupervised training regime can use subjective class labels to achieve similar results to using the ground truth labels, opening the door to much wider applicability of these techniques as expensive ground truth labels (provenance) of stimuli data can be replaced by crowd source evaluations (subjective labels). Overall, we showed that neural networks can be used on subjective data, which includes observer perceptions of the doubt felt by the presenters of information. Our ability to detect this doubt effect is due to our observers’ underlying emotional reactions to what they see, reflected in their physiological signals, and learnt by our neural networks. This kind of technology using physiological signals collected in real time from observers could be used to reflect audience distrust, and perhaps could lead to increased truthfulness in statements presented via the Media.

Original languageEnglish
Title of host publicationNeural Information Processing - 25th International Conference, ICONIP 2018, Proceedings
EditorsSeiichi Ozawa, Andrew Chi Sing Leung, Long Cheng
PublisherSpringer-Verlag
Pages610-621
Number of pages12
ISBN (Print)9783030042110
DOIs
Publication statusPublished - Jan 1 2018
Event25th International Conference on Neural Information Processing, ICONIP 2018 - Siem Reap, Cambodia
Duration: Dec 13 2018Dec 16 2018

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11304 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other25th International Conference on Neural Information Processing, ICONIP 2018
CountryCambodia
CitySiem Reap
Period12/13/1812/16/18

Fingerprint

Belief Networks
Labels
Observer
Neural Networks
Neural networks
Subjective Evaluation
Provenance
Predictability
High Accuracy

Keywords

  • Doubt
  • Information veracity
  • Neural networks
  • Pupillary responses
  • Semi-unsupervised training
  • Subjective belief
  • Trust

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Zhu, X., Qin, Z., Gedeon, T., Jones, R., Hossain, M. Z., & Caldwell, S. (2018). Detecting the doubt effect and subjective beliefs using neural networks and observers’ pupillary responses. In S. Ozawa, A. C. S. Leung, & L. Cheng (Eds.), Neural Information Processing - 25th International Conference, ICONIP 2018, Proceedings (pp. 610-621). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11304 LNCS). Springer-Verlag. https://doi.org/10.1007/978-3-030-04212-7_54

Detecting the doubt effect and subjective beliefs using neural networks and observers’ pupillary responses. / Zhu, Xuanying; Qin, Zhenyue; Gedeon, Tom; Jones, Richard; Hossain, Md. Zakir; Caldwell, Sabrina.

Neural Information Processing - 25th International Conference, ICONIP 2018, Proceedings. ed. / Seiichi Ozawa; Andrew Chi Sing Leung; Long Cheng. Springer-Verlag, 2018. p. 610-621 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11304 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Zhu, X, Qin, Z, Gedeon, T, Jones, R, Hossain, MZ & Caldwell, S 2018, Detecting the doubt effect and subjective beliefs using neural networks and observers’ pupillary responses. in S Ozawa, ACS Leung & L Cheng (eds), Neural Information Processing - 25th International Conference, ICONIP 2018, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11304 LNCS, Springer-Verlag, pp. 610-621, 25th International Conference on Neural Information Processing, ICONIP 2018, Siem Reap, Cambodia, 12/13/18. https://doi.org/10.1007/978-3-030-04212-7_54
Zhu X, Qin Z, Gedeon T, Jones R, Hossain MZ, Caldwell S. Detecting the doubt effect and subjective beliefs using neural networks and observers’ pupillary responses. In Ozawa S, Leung ACS, Cheng L, editors, Neural Information Processing - 25th International Conference, ICONIP 2018, Proceedings. Springer-Verlag. 2018. p. 610-621. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-030-04212-7_54
Zhu, Xuanying ; Qin, Zhenyue ; Gedeon, Tom ; Jones, Richard ; Hossain, Md. Zakir ; Caldwell, Sabrina. / Detecting the doubt effect and subjective beliefs using neural networks and observers’ pupillary responses. Neural Information Processing - 25th International Conference, ICONIP 2018, Proceedings. editor / Seiichi Ozawa ; Andrew Chi Sing Leung ; Long Cheng. Springer-Verlag, 2018. pp. 610-621 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{f1476d556fbc4a32a3fe0961638577df,
title = "Detecting the doubt effect and subjective beliefs using neural networks and observers’ pupillary responses",
abstract = "We investigated the physiological underpinnings to detect the ‘doubt effect’ – where a presenter’s subjective belief in some information has been manipulated. We constructed stimulus videos in which presenters delivered information that in some cases they were led to doubt, but asked to “present anyway”. We then showed these stimuli to observers and measured their physiological signals (pupillary responses). Neural networks trained with two statistical features reached a higher accuracy in differentiating the doubt/ manipulated-belief compared to the observers’ own veracity judgments, which is overall at chance level. We also trained confirmatory neural networks for the predictability of specific stimuli and extracted significant information on those stimulus presenters. We further showed that a semi-unsupervised training regime can use subjective class labels to achieve similar results to using the ground truth labels, opening the door to much wider applicability of these techniques as expensive ground truth labels (provenance) of stimuli data can be replaced by crowd source evaluations (subjective labels). Overall, we showed that neural networks can be used on subjective data, which includes observer perceptions of the doubt felt by the presenters of information. Our ability to detect this doubt effect is due to our observers’ underlying emotional reactions to what they see, reflected in their physiological signals, and learnt by our neural networks. This kind of technology using physiological signals collected in real time from observers could be used to reflect audience distrust, and perhaps could lead to increased truthfulness in statements presented via the Media.",
keywords = "Doubt, Information veracity, Neural networks, Pupillary responses, Semi-unsupervised training, Subjective belief, Trust",
author = "Xuanying Zhu and Zhenyue Qin and Tom Gedeon and Richard Jones and Hossain, {Md. Zakir} and Sabrina Caldwell",
year = "2018",
month = "1",
day = "1",
doi = "10.1007/978-3-030-04212-7_54",
language = "English",
isbn = "9783030042110",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer-Verlag",
pages = "610--621",
editor = "Seiichi Ozawa and Leung, {Andrew Chi Sing} and Long Cheng",
booktitle = "Neural Information Processing - 25th International Conference, ICONIP 2018, Proceedings",

}

TY - GEN

T1 - Detecting the doubt effect and subjective beliefs using neural networks and observers’ pupillary responses

AU - Zhu, Xuanying

AU - Qin, Zhenyue

AU - Gedeon, Tom

AU - Jones, Richard

AU - Hossain, Md. Zakir

AU - Caldwell, Sabrina

PY - 2018/1/1

Y1 - 2018/1/1

N2 - We investigated the physiological underpinnings to detect the ‘doubt effect’ – where a presenter’s subjective belief in some information has been manipulated. We constructed stimulus videos in which presenters delivered information that in some cases they were led to doubt, but asked to “present anyway”. We then showed these stimuli to observers and measured their physiological signals (pupillary responses). Neural networks trained with two statistical features reached a higher accuracy in differentiating the doubt/ manipulated-belief compared to the observers’ own veracity judgments, which is overall at chance level. We also trained confirmatory neural networks for the predictability of specific stimuli and extracted significant information on those stimulus presenters. We further showed that a semi-unsupervised training regime can use subjective class labels to achieve similar results to using the ground truth labels, opening the door to much wider applicability of these techniques as expensive ground truth labels (provenance) of stimuli data can be replaced by crowd source evaluations (subjective labels). Overall, we showed that neural networks can be used on subjective data, which includes observer perceptions of the doubt felt by the presenters of information. Our ability to detect this doubt effect is due to our observers’ underlying emotional reactions to what they see, reflected in their physiological signals, and learnt by our neural networks. This kind of technology using physiological signals collected in real time from observers could be used to reflect audience distrust, and perhaps could lead to increased truthfulness in statements presented via the Media.

AB - We investigated the physiological underpinnings to detect the ‘doubt effect’ – where a presenter’s subjective belief in some information has been manipulated. We constructed stimulus videos in which presenters delivered information that in some cases they were led to doubt, but asked to “present anyway”. We then showed these stimuli to observers and measured their physiological signals (pupillary responses). Neural networks trained with two statistical features reached a higher accuracy in differentiating the doubt/ manipulated-belief compared to the observers’ own veracity judgments, which is overall at chance level. We also trained confirmatory neural networks for the predictability of specific stimuli and extracted significant information on those stimulus presenters. We further showed that a semi-unsupervised training regime can use subjective class labels to achieve similar results to using the ground truth labels, opening the door to much wider applicability of these techniques as expensive ground truth labels (provenance) of stimuli data can be replaced by crowd source evaluations (subjective labels). Overall, we showed that neural networks can be used on subjective data, which includes observer perceptions of the doubt felt by the presenters of information. Our ability to detect this doubt effect is due to our observers’ underlying emotional reactions to what they see, reflected in their physiological signals, and learnt by our neural networks. This kind of technology using physiological signals collected in real time from observers could be used to reflect audience distrust, and perhaps could lead to increased truthfulness in statements presented via the Media.

KW - Doubt

KW - Information veracity

KW - Neural networks

KW - Pupillary responses

KW - Semi-unsupervised training

KW - Subjective belief

KW - Trust

UR - http://www.scopus.com/inward/record.url?scp=85059012296&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85059012296&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-04212-7_54

DO - 10.1007/978-3-030-04212-7_54

M3 - Conference contribution

SN - 9783030042110

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 610

EP - 621

BT - Neural Information Processing - 25th International Conference, ICONIP 2018, Proceedings

A2 - Ozawa, Seiichi

A2 - Leung, Andrew Chi Sing

A2 - Cheng, Long

PB - Springer-Verlag

ER -