Abstract
We describe an experiment carried out using a French version of CALL-SLT, a web-enabled CALL game in which students at each turn are prompted to give a semi-free spoken response which the system then either accepts or rejects. The central question we investigate is whether the response is appropriate; we do this by extracting pairs of utterances where both members of the pair are responses by the same student to the same prompt, and where one response is accepted and one rejected. When the two spoken responses are presented in random order, native speakers show a reasonable degree of agreement in judging that the accepted utterance is better than the rejected one. We discuss the significance of the results and also present a small study supporting the claim that native speakers are nearly always recognised by the system, while non-native speakers are rejected a significant proportion of the time.- Anthology ID:
- L12-1225
- Volume:
- Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12)
- Month:
- May
- Year:
- 2012
- Address:
- Istanbul, Turkey
- Editors:
- Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Mehmet Uğur Doğan, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, Stelios Piperidis
- Venue:
- LREC
- SIG:
- Publisher:
- European Language Resources Association (ELRA)
- Note:
- Pages:
- 2690–2694
- Language:
- URL:
- http://www.lrec-conf.org/proceedings/lrec2012/pdf/434_Paper.pdf
- DOI:
- Cite (ACL):
- Manny Rayner, Pierrette Bouillon, and Johanna Gerlach. 2012. Evaluating Appropriateness Of System Responses In A Spoken CALL Game. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12), pages 2690–2694, Istanbul, Turkey. European Language Resources Association (ELRA).
- Cite (Informal):
- Evaluating Appropriateness Of System Responses In A Spoken CALL Game (Rayner et al., LREC 2012)
- PDF:
- http://www.lrec-conf.org/proceedings/lrec2012/pdf/434_Paper.pdf