Overview of the IWSLT 2007 evaluation campaign

Cameron Shaw Fordyce


Abstract
In this paper we give an overview of the 2007 evaluation campaign for the International Workshop on Spoken Language Translation (IWSLT)1. As with previous evaluation campaigns, the primary focus of the workshop was the translation of spoken language in the travel domain. This year there were four language pairs; the translation of Chinese, Italian, Arabic, and Japanese into English. The input data consisted of the output of ASR systems for read speech and clean text. The exceptions were the challenge task of the Italian English language pair which used spontaneous speech ASR outputs and transcriptions and the Chinese English task which used only clean text. A new characteristic of this year’s evaluation campaign was an increased focus on the sharing of resources. Participants were requested to submit the data and supplementary resources used in building their systems so that the other participants might be able to take advantage of the same resources. A second new characteristic this year was the focus on the human evaluation of systems. Each primary run was judged in the human evaluation for every task using a straightforward ranking of systems. This year's workshop saw an increased participation over last year's workshop. This year 24 groups submitted runs to one or more of the tasks, compared to the 19 groups that submitted runs last year [1]. Automatic and human evaluation were carried out to measure MT performance under each condition, ASR system outputs for read speech, spontaneous travel dialogues, and clean text.
Anthology ID:
2007.iwslt-1.1
Volume:
Proceedings of the Fourth International Workshop on Spoken Language Translation
Month:
October 15-16
Year:
2007
Address:
Trento, Italy
Venue:
IWSLT
SIG:
SIGSLT
Publisher:
Note:
Pages:
Language:
URL:
https://aclanthology.org/2007.iwslt-1.1
DOI:
Bibkey:
Cite (ACL):
Cameron Shaw Fordyce. 2007. Overview of the IWSLT 2007 evaluation campaign. In Proceedings of the Fourth International Workshop on Spoken Language Translation, Trento, Italy.
Cite (Informal):
Overview of the IWSLT 2007 evaluation campaign (Fordyce, IWSLT 2007)
Copy Citation:
PDF:
https://preview.aclanthology.org/paclic-22-ingestion/2007.iwslt-1.1.pdf