AI-Josyu: Thinking Support System in Class by Real-time Speech Recognition and Keyword Extraction
Abstract
In this paper, we present a thinking support system, AI-Josyu. This system also operates as a class support system which helps to teachers for lightening their work. AI-Josyu is implemented based on media-driven real-time content management framework. The system links real world media and legacy media contents together. In resent years, it is easier to collect a large amount of various kinds of data which are created with sensors in the real world. The system realizes interconnection and utilization of legacy media contents. The legacy media contents are generated and scattered on the Internet. The framework has four modules, which are called “acquisition,†“extraction,†“selection,†and “retrieval.†The real world media and the legacy media contents are interconnected by these modules. This interconnection includes semantic components. This system records teacher's voice of its lecture in real time and presents retrieved legacy media contents corresponding to subject of the lecture. By this presentation, preparing of the legacy contents is not required. This system automatically retrieves and shows the legacy media contents. This system helps students to understand contents of the lecture. In addition, the system attends to expansion of ideas. We constructed the system and conducted the demonstration in class. It shows that the system is helpful to teacher and students for expansion of thinking.
Downloads
References
Ossama Abdel-Hamid, Abdel-rahman Mohamed, Hui Jiang, Li Deng, Gerald Penn, and Dong Yu, Convolutional Neural Networks for Speech Recognition, IEEE/ACM Transactions on Audio, Speech, and Language Processing, Vol. 22, No. 10, pp. 1533-1545, 2014.
Takashi Kitagawa, Takafumi Nakanishi, and Yasushi Kiyoki, An implementation method of automatic metadata extraction method for music data and its application to semantic associative search, Systems and Computers in Japan, Vol. 35(6), pp. 59-78, 2004.
Takashi Kitagawa, Takafumi Nakanishi, and Yasushi Kiyoki, An Implementation Method of Automatic Metadata Extraction Method for Image Data and its Application to a Semantic Associative Search, Information Processing Society of Japan Transactions on Databases, Vol.43(SIG12) (TOD16), pp. 38-51, 2002.
Takafumi Nakanishi, Ryotaro Okada, and Takashi Kitagawa, Automatic Media Content Creation System According to an Impression by Recognition-Creation Operators, Proceedings of the 2016 IEEE/ACIS 15th International Conference on Computer and Information Science (ICIS), pp. 1-6, 2016.
Takafumi Nakanishi, Kyohei Matsumoto, Toshitada Sakawa, Kengo Onodera, Shinichiro Orimo, and Hiroyuki Kobayashi, Media-driven Real-time Content Manegement Framework and its Application to In-Class Thinking Support System, Proceedings of the 2018 International Electronics Symposium on Knowledge Creation and Intelligent Computing (IES-KCIC), Surabaya, East Java, Indonesia, pp. 274-279, 2018.
Keith Bain, Sara Basson, Alexander Faisman, and Dimitri Kanevsky, Accessibility, transcription, and access everywhere, IBM Systems Journal, Vol.44, No.3, pp.589-604, 2005.
Vivien Arief Wardhany, Sritrusta Sukaridhoto, and Amang Sudarsono, Indonesian Automatic Speech Recognition For Command Speech Controller Multimedia Player, EMITTER International Journal of Engineering Technology, Vol.2, No.2, pp39-48, 2014.
Rohit Ranchal, Teresa Taber-Doughty, Yiren Guo, Keith Bain, Heather Martin, J. Paul Robinson, and Bradley S. Duerstock, Using speech recognition for real-time captioning and lecture transcription in the classroom, IEEE Transactions on Learning Technologies, Vol.6, No.4, pp. 299-311, 2013.
Joseph Robison, and Carl Jensema, Computer Speech Recognition as an Assistive Device for Deaf and Hard of Hearing People, Challenge of Change: Beyond the Horizon, Proceedings of Seventh Biennial Conference Postsecondary Education Persons Who Are Deaf or Hard of Hearing, 1996.
Sandy Watson, and Linda Johnston, Assistive Technology in the Inclusive Science Classroom: Devices and services can help science students with a wide variety of needs, The Science Teacher, Vol. 74, No. 3, pp. 34-38, 2007.
Mike Wald, Captioning for Deaf and Hard of Hearing People by Editing Automatic Speech Recognition in Real Time, Proceedings of 10th International Conference on Computers Helping People with Special Needs ICCHP 2006, pp. 683-690, 2006.
Petr Červa, Jan Silovský, Jindrich Zdánský, J. Nouza, and Jiřà Málek, Real-time lecture transcription using ASR for Czech hearing impaired or deaf students, Proceedings of INTERSPEECH 2012, 13th Annual Conference of the ISCA International Speech Communication Association, Portland, OR, USA, 2012.
Tatsuya Kawahara, Norihiro Katsumaru, Yuya Akita, and Shinsuke Mori, Classroom Note-taking System for Hearing Impaired Students using Automatic Speech Recognition Adapted to Lectures, Proceedings of INTERSPEECH 2010, 11th Annual Conference of the International Speech Communication Association, Makuhari, Chiba, Japan, pp. 626-629, 2010.
Margaret Martyn, Clickers in the Classroom: An Active Learning Approach, EDUCAUSE Quarterly Magazine, Vol. 30, No. 2, pp.71-74, 2007.
April R. Trees, and Michele H. Jackson, The learning environment in clicker classrooms: student processes of learning and involvement in large university-level courses using student response systems, Learning, Media and Technology, Vol. 32, No.1, pp. 21–40, 2007.
Motoki Yokoyama, Yasushi Kiyoki, and Tetsuya Mita, A Similarity-Ranking Method on Semantic Computing for Providing Information-Services in Station-Concierge System, EMITTER International Journal of Engineering Technology, Vol.5, No.1, pp16-35, 2017.
Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig, Linguistic Regularities in Continuous Space Word Representations, Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT-2013), pp. 746-751, 2013.
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean, Efficient Estimation of Word Representations in Vector Space, Computing Research Repository (CoRR), 2013.
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean, Distributed Representations of Words and Phrases and their Compositionality, Advances in Neural Information Processing Systems 26, pp. 3111-3119, Curran Associates, Inc. (New York), 2013.
Copyright (c) 2019 EMITTER International Journal of Engineering Technology
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
The copyright to this article is transferred to Politeknik Elektronika Negeri Surabaya(PENS) if and when the article is accepted for publication. The undersigned hereby transfers any and all rights in and to the paper including without limitation all copyrights to PENS. The undersigned hereby represents and warrants that the paper is original and that he/she is the author of the paper, except for material that is clearly identified as to its original source, with permission notices from the copyright owners where required. The undersigned represents that he/she has the power and authority to make and execute this assignment. The copyright transfer form can be downloaded here .
The corresponding author signs for and accepts responsibility for releasing this material on behalf of any and all co-authors. This agreement is to be signed by at least one of the authors who have obtained the assent of the co-author(s) where applicable. After submission of this agreement signed by the corresponding author, changes of authorship or in the order of the authors listed will not be accepted.
Retained Rights/Terms and Conditions
- Authors retain all proprietary rights in any process, procedure, or article of manufacture described in the Work.
- Authors may reproduce or authorize others to reproduce the work or derivative works for the author’s personal use or company use, provided that the source and the copyright notice of Politeknik Elektronika Negeri Surabaya (PENS) publisher are indicated.
- Authors are allowed to use and reuse their articles under the same CC-BY-NC-SA license as third parties.
- Third-parties are allowed to share and adapt the publication work for all non-commercial purposes and if they remix, transform, or build upon the material, they must distribute under the same license as the original.
Plagiarism Check
To avoid plagiarism activities, the manuscript will be checked twice by the Editorial Board of the EMITTER International Journal of Engineering Technology (EMITTER Journal) using iThenticate Plagiarism Checker and the CrossCheck plagiarism screening service. The similarity score of a manuscript has should be less than 25%. The manuscript that plagiarizes another author’s work or author's own will be rejected by EMITTER Journal.
Authors are expected to comply with EMITTER Journal's plagiarism rules by downloading and signing the plagiarism declaration form here and resubmitting the form, along with the copyright transfer form via online submission.
