Fumiaki Sugaya


2021

pdf
Field Experiments of Real Time Foreign News Distribution Powered by MT
Keiji Yasuda | Ichiro Yamada | Naoaki Okazaki | Hideki Tanaka | Hidehiro Asaka | Takeshi Anzai | Fumiaki Sugaya
Proceedings of Machine Translation Summit XVIII: Users and Providers Track

Field experiments on a foreign news distribution system using two key technologies are reported. The first technology is a summarization component, which is used for generating news headlines. This component is a transformer-based abstractive text summarization system which is trained to output headlines from the leading sentences of news articles. The second technology is machine translation (MT), which enables users to read foreign news articles in their mother language. Since the system uses MT, users can immediately access the latest foreign news. 139 Japanese LINE users participated in the field experiments for two weeks, viewing about 40,000 articles which had been translated from English to Japanese. We carried out surveys both during and after the experiments. According to the results, 79.3% of users evaluated the headlines as adequate, while 74.7% of users evaluated the automatically translated articles as intelligible. According to the post-experiment survey, 59.7% of users wished to continue using the system; 11.5% of users did not. We also report several statistics of the experiments.

2006

pdf
Using the Web to Disambiguate Acronyms
Eiichiro Sumita | Fumiaki Sugaya
Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers

pdf
Word Pronunciation Disambiguation using the Web
Eiichiro Sumita | Fumiaki Sugaya
Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers

2005

pdf
Measuring Non-native Speakers’ Proficiency of English by Using a Test with Automatically-Generated Fill-in-the-Blank Questions
Eiichiro Sumita | Fumiaki Sugaya | Seiichi Yamamoto
Proceedings of the Second Workshop on Building Educational Applications Using NLP

2004

pdf
Automatic Measuring of English Language Proficiency using MT Evaluation Technology
Keiji Yasuda | Fumiaki Sugaya | Eiichiro Sumita | Toshiyuki Takezawa | Genichiro Kikui | Seiichi Yamamoto
Proceedings of the Workshop on eLearning for Computational Linguistics and Computational Linguistics for eLearning

2003

pdf
Applications of Automatic Evaluation Methods to Measuring a Capability of Speech Translation System
Keiji Yasuda | Fumiaki Sugaya | Toshiyuki Takezawa | Seiichi Yamamoto | Masuzo Yanagida
10th Conference of the European Chapter of the Association for Computational Linguistics

2002

pdf
Automatic machine translation selection scheme to output the best result
Keiji Yasuda | Fumiaki Sugaya | Toshiyuki Takezawa | Seiichi Yamamoto | Masuzo Yanagida
Proceedings of the Third International Conference on Language Resources and Evaluation (LREC’02)

pdf
Toward a Broad-coverage Bilingual Corpus for Speech Translation of Travel Conversations in the Real World
Toshiyuki Takezawa | Eiichiro Sumita | Fumiaki Sugaya | Hirofumi Yamamoto | Seiichi Yamamoto
Proceedings of the Third International Conference on Language Resources and Evaluation (LREC’02)

pdf
Quality-Sensitive Test Set Selection for a Speech Translation System
Fumiaki Sugaya | Keiji Yasuda | Toshiyuki Takezawa | Seiichi Yamamoto
Proceedings of the ACL-02 Workshop on Speech-to-Speech Translation: Algorithms and Systems

2001

pdf
Precise measurement method of a speech translation system’s capability with a paired comparison method between the system and humans
Fumiaki Sugaya | Keiji Yasuda | Toshiyuki Takezawa | Seiichi Yamamoto
Proceedings of Machine Translation Summit VIII

The main goal of the present paper is to propose new schemes for the overall evaluation of a speech translation system. These schemes are expected to support and improve the design of the target application system, and precisely determine its performance. Experiments are conducted on the Japanese-to-English speech translation system ATR-MATRIX, which was developed at ATR Interpreting Telecommunications Research Laboratories. In the proposed schemes, the system’s translations are compared with those of a native Japanese taking the Test of English for International Communication (TOEIC), which is used as a measure of one’s speech translation capability. Subjective and automatic comparisons are made and the results are compared. A regression analysis on the subjective results shows that the speech translation capability of ATR-MATRIX matches a Japanese person scoring around 500 on the TOEIC. The automatic comparisons also show promising results.

pdf
An automatic evaluation method of translation quality using translation answer candidates queried from a parallel corpus
Keiji Yasuda | Fumiaki Sugaya | Toshiyuki Takezawa | Seiichi Yamamoto | Masuzo Yanagida
Proceedings of Machine Translation Summit VIII

An automatic translation quality evaluation method is proposed. In the proposed method, a parallel corpus is used to query translation answer candidates. The translation output is evaluated by measuring the similarity between the translation output and translation answer candidates with DP matching. This method evaluates a language translation subsystem of the Japanese-to-English ATR-MATRIX speech translation system developed at ATR Interpreting Telecommunications Research Laboratories. Discriminant analysis is then carried out to examine the evaluation performance of the proposed method. Experimental results show the effectiveness of the proposed method. The discriminant ratio is 83.5% for 2-class discrimination between absolutely correct and less appropriate translations classified subjectively. Also discussed are issues of the proposed method when it is applied to speech translation systems which inevitably make recognition errors.

1999

pdf
A new evaluation method for speech translation systems and a case study on ATR-MATRIX from Japanese to English
Toshiyuki Takezawa | Fumiaki Sugaya | Akio Yokoo | Seiichi Yamamoto
Proceedings of Machine Translation Summit VII

ATR-MATRIX is a multi-lingual speech-to-speech translation system designed to facilitate communications between two parties of different languages engaged in a spontaneous conversation in a travel arrangement domain. In this paper, we propose a new evaluation method for speech translation systems. Our current focus is on measuring the robustness of a language translation sub-system, with quick calculation and low cost. Therefore, we calculate the difference between the translation output from transcription texts and the translation output from input speech by a dynamic programming method. We present the first trial experiment of this method applied to our Japanese-to-English speech translation system. We also provide related discussions on such points as error analysis and the relationship between the proposed method and translation quality evaluation manually done by humans.