Toggle navigation
BIP!
Services
Finder
Readings
Scholar
Data, API & Code
Indicators
About
Help
Log In
Clear all
2018 •
Overview of the 2018 spoken CALL shared task
Authors:
Claudia Baur, Andrew Caines, Cathy Chua, Johanna Gerlach, Mengjie Qian, Manny Rayner, Martin J. Russell, Helmer Strik, Xizi Wei
Abstract:
© 2018 International Speech Communication Association. All rights reserved. We present an overview of the second edition of the Spoken CALL Shared Task. Groups competed on a prompt-response task using English-language data collected, through an online CALL game, from Swiss German teens in their second and third years of learning English. Each item consists of a written German prompt and an audio file containing a spoken response. The task is to accept linguistically correct responses and reject linguistically incorrect ones, with “linguistic (...)
© 2018 International Speech Communication Association. All rights reserved. We present an overview of the second edition of the Spoken CALL Shared Task. Groups competed on a prompt-response task using English-language data collected, through an online CALL game, from Swiss German teens in their second and third years of learning English. Each item consists of a written German prompt and an audio file containing a spoken response. The task is to accept linguistically correct responses and reject linguistically incorrect ones, with “linguistically correct” defined by a gold standard derived from human annotations. Scoring was performed using a metric defined as the ratio of the relative rejection rates on incorrect and correct responses. The second edition received eighteen entries and showed very substantial improvement on the first edition; all entries were better than the best entry from the first edition, and the best score was about four times higher. We present the task, the resources, the results, a discussion of the metrics used, and an analysis of what makes items challenging. In particular, we present quantitative evidence suggesting that incorrect responses are much more difficult to process than correct responses, and that the most significant factor in making a response challenging is its distance from the closest training example.
(Read More)
External links:
OpenAIRE
Works in BIP! associated with the specified OpenAIRE research product
Overview of the 2018 Spoken CALL Shared Task
Claudia Baur, Andrew Caines, Cathy Chua, Johanna Gerlach, Mengjie Qian, Manny Rayner, Martin Russell (...)
Interspeech 2018
·
2018
Natural language processing
|
Artificial intelligence
|
Speech recognition
|
Missing works (1)
Missing works
10.17863/cam.93398
We have placed cookies on your device to help make this website and the services we offer better. By using this site, you agree to the use of cookies.
Learn more
I accept
Are you sure you want to delete this bookmark ?