Automated inference of cognitive performance by fusing multimodal information acquired by smartphone

Takashi Hamatani; Keiichi Ochiai; Akiya Inagaki; Naoki Yamamoto; Yusuke Fukazawa; Masatoshi Kimoto; Kazuki Kiriu; Kouhei Kaminishi; Jun Ota; Yuri Terasawa; Tsukasa Okimura; Takaki Maeda

doi:10.1145/3341162.3346275

Automated inference of cognitive performance by fusing multimodal information acquired by smartphone

Takashi Hamatani, Keiichi Ochiai, Akiya Inagaki, Naoki Yamamoto, Yusuke Fukazawa, Masatoshi Kimoto, Kazuki Kiriu, Kouhei Kaminishi, Jun Ota, Yuri Terasawa, Tsukasa Okimura, Takaki Maeda

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

2 Citations (Scopus)

Abstract

Recognizing human cognitive performance is important for preserving working efficiency and preventing human error. This paper presents a method for estimating cognitive performance by leveraging multiple information available in a smartphone. The method employs the Go-NoGo task to measure cognitive performance, and fuses contextual and behavioral features to identify the level of performance. It was confirmed that the proposed method could recognize whether cognitive performance was high or low with an average accuracy of 71%, even when only referring to inertial sensor logs. Combining sensing modalities improved the accuracy up to 74%.

Original language	English
Title of host publication	UbiComp/ISWC 2019- - Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers
Publisher	Association for Computing Machinery, Inc
Pages	921-928
Number of pages	8
ISBN (Electronic)	9781450368698
DOIs	https://doi.org/10.1145/3341162.3346275
Publication status	Published - 2019 Sept 9
Event	2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and 2019 ACM International Symposium on Wearable Computers, UbiComp/ISWC 2019 - London, United Kingdom Duration: 2019 Sept 9 → 2019 Sept 13

Publication series

Name	UbiComp/ISWC 2019- - Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers

Conference

Conference	2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and 2019 ACM International Symposium on Wearable Computers, UbiComp/ISWC 2019
Country/Territory	United Kingdom
City	London
Period	19/9/9 → 19/9/13

Keywords

Cognitive performance
Go-NoGo task
Machine learning
Smartphone log

ASJC Scopus subject areas

Software
Hardware and Architecture
Computer Networks and Communications

Access to Document

10.1145/3341162.3346275

Cite this

Hamatani, T., Ochiai, K., Inagaki, A., Yamamoto, N., Fukazawa, Y., Kimoto, M., Kiriu, K., Kaminishi, K., Ota, J., Terasawa, Y., Okimura, T., & Maeda, T. (2019). Automated inference of cognitive performance by fusing multimodal information acquired by smartphone. In UbiComp/ISWC 2019- - Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers (pp. 921-928). (UbiComp/ISWC 2019- - Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers). Association for Computing Machinery, Inc. https://doi.org/10.1145/3341162.3346275

Automated inference of cognitive performance by fusing multimodal information acquired by smartphone. / Hamatani, Takashi; Ochiai, Keiichi; Inagaki, Akiya et al.
UbiComp/ISWC 2019- - Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers. Association for Computing Machinery, Inc, 2019. p. 921-928 (UbiComp/ISWC 2019- - Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Hamatani, T, Ochiai, K, Inagaki, A, Yamamoto, N, Fukazawa, Y, Kimoto, M, Kiriu, K, Kaminishi, K, Ota, J, Terasawa, Y, Okimura, T & Maeda, T 2019, Automated inference of cognitive performance by fusing multimodal information acquired by smartphone. in UbiComp/ISWC 2019- - Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers. UbiComp/ISWC 2019- - Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers, Association for Computing Machinery, Inc, pp. 921-928, 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and 2019 ACM International Symposium on Wearable Computers, UbiComp/ISWC 2019, London, United Kingdom, 19/9/9. https://doi.org/10.1145/3341162.3346275

Hamatani T, Ochiai K, Inagaki A, Yamamoto N, Fukazawa Y, Kimoto M et al. Automated inference of cognitive performance by fusing multimodal information acquired by smartphone. In UbiComp/ISWC 2019- - Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers. Association for Computing Machinery, Inc. 2019. p. 921-928. (UbiComp/ISWC 2019- - Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers). doi: 10.1145/3341162.3346275

Hamatani, Takashi ; Ochiai, Keiichi ; Inagaki, Akiya et al. / Automated inference of cognitive performance by fusing multimodal information acquired by smartphone. UbiComp/ISWC 2019- - Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers. Association for Computing Machinery, Inc, 2019. pp. 921-928 (UbiComp/ISWC 2019- - Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers).

@inproceedings{2444720b91884c57a68c223addbdadcf,

title = "Automated inference of cognitive performance by fusing multimodal information acquired by smartphone",

abstract = "Recognizing human cognitive performance is important for preserving working efficiency and preventing human error. This paper presents a method for estimating cognitive performance by leveraging multiple information available in a smartphone. The method employs the Go-NoGo task to measure cognitive performance, and fuses contextual and behavioral features to identify the level of performance. It was confirmed that the proposed method could recognize whether cognitive performance was high or low with an average accuracy of 71%, even when only referring to inertial sensor logs. Combining sensing modalities improved the accuracy up to 74%.",

keywords = "Cognitive performance, Go-NoGo task, Machine learning, Smartphone log",

author = "Takashi Hamatani and Keiichi Ochiai and Akiya Inagaki and Naoki Yamamoto and Yusuke Fukazawa and Masatoshi Kimoto and Kazuki Kiriu and Kouhei Kaminishi and Jun Ota and Yuri Terasawa and Tsukasa Okimura and Takaki Maeda",

note = "Publisher Copyright: {\textcopyright} 2019 Association for Computing Machinery.; 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and 2019 ACM International Symposium on Wearable Computers, UbiComp/ISWC 2019 ; Conference date: 09-09-2019 Through 13-09-2019",

year = "2019",

month = sep,

day = "9",

doi = "10.1145/3341162.3346275",

language = "English",

series = "UbiComp/ISWC 2019- - Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers",

publisher = "Association for Computing Machinery, Inc",

pages = "921--928",

booktitle = "UbiComp/ISWC 2019- - Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers",

}

TY - GEN

T1 - Automated inference of cognitive performance by fusing multimodal information acquired by smartphone

AU - Hamatani, Takashi

AU - Ochiai, Keiichi

AU - Inagaki, Akiya

AU - Yamamoto, Naoki

AU - Fukazawa, Yusuke

AU - Kimoto, Masatoshi

AU - Kiriu, Kazuki

AU - Kaminishi, Kouhei

AU - Ota, Jun

AU - Terasawa, Yuri

AU - Okimura, Tsukasa

AU - Maeda, Takaki

PY - 2019/9/9

Y1 - 2019/9/9

N2 - Recognizing human cognitive performance is important for preserving working efficiency and preventing human error. This paper presents a method for estimating cognitive performance by leveraging multiple information available in a smartphone. The method employs the Go-NoGo task to measure cognitive performance, and fuses contextual and behavioral features to identify the level of performance. It was confirmed that the proposed method could recognize whether cognitive performance was high or low with an average accuracy of 71%, even when only referring to inertial sensor logs. Combining sensing modalities improved the accuracy up to 74%.

AB - Recognizing human cognitive performance is important for preserving working efficiency and preventing human error. This paper presents a method for estimating cognitive performance by leveraging multiple information available in a smartphone. The method employs the Go-NoGo task to measure cognitive performance, and fuses contextual and behavioral features to identify the level of performance. It was confirmed that the proposed method could recognize whether cognitive performance was high or low with an average accuracy of 71%, even when only referring to inertial sensor logs. Combining sensing modalities improved the accuracy up to 74%.

KW - Cognitive performance

KW - Go-NoGo task

KW - Machine learning

KW - Smartphone log

UR - http://www.scopus.com/inward/record.url?scp=85072881540&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85072881540&partnerID=8YFLogxK

U2 - 10.1145/3341162.3346275

DO - 10.1145/3341162.3346275

M3 - Conference contribution

AN - SCOPUS:85072881540

T3 - UbiComp/ISWC 2019- - Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers

SP - 921

EP - 928

BT - UbiComp/ISWC 2019- - Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers

PB - Association for Computing Machinery, Inc

T2 - 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and 2019 ACM International Symposium on Wearable Computers, UbiComp/ISWC 2019

Y2 - 9 September 2019 through 13 September 2019

ER -

Automated inference of cognitive performance by fusing multimodal information acquired by smartphone

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this