============================================================================
                            REVIEWER #1
============================================================================

Detailed Comments
---------------------------------------------------------------------------
This paper presents a tool for aiding users in writing translations in a language they do not know (outbound translation).

The tool consists of a user interface where a text in the source language (known by the user) can be entered, an automatic translation in the unknown language is provided, quality estimation of translated words/phrases is available as well as word alignment between source and target words and, finally, back translation in the source language is provided as well. This way, the user is allowed to decide if the translation is reasonable and, in case, to rephrase her input for improving the final results.

The tool and its components (MT, QE, word/sentence aligner) are described. Moreover, a number of experiments was performed and dpresented here with the goal of assessing the usefulness of the tool and to have a first insights on the strategies followed by users to accomplish the outbound translation.

The paper and the tool described in it are interesting for LREC and the research community. The manuscript is in general clear and well written, even if some statements should be made clearer (see below). The validation of the quality of the tool is solid. Bibliography is adequate.

Minor suggestions

(FIXED)
General: I suggest to check the over-usage of "comma". For example usually it has not to be put before pronouns which/that, like you do in "to languages, which" and "assurance, that" (Section 1), "final inputs, which" and "the segments, which" (Section 5.2), "noting, that" (Section 5.3). 

(FIXED)
Section 3.2
- on the text between the first and the second -> on the texts in the first and the second

(FIXED)
Section 3.3
- models made obvious mistakes -> mistakes are never "obvious". Please, be more precise.

(FIXED)
Section 3.4.1
- e. g. -> e.g. [delete space]

(FIXED)
Section 3.5
- their ease of use and performance -> their ease of use and good performance

(FIXED)
Section 4
- was enabled only for the first section -> "sections" of experiments as listed in Sections 4.2.1, 4.2.2 and 4.2.3 have not been introduced/described yet, then it's hard for the reader to understand what you are stating here. Please, rephrase.

(FIXED)
Section 4.1
- these groups as bilingual and monolingual respectively -> these groups as monolingual and bilingual, respectively.

(FIXED)
Section 5.2
- We then focused [...] Table 8. -> this paragraph is not clear, in particular I do not well understand the definition of "first viable" input. But even  the first sentence is quite misleading where you write that this paragraph regards "segments which were further edited". "Further" with respect to what? In the preceeding paragraph it seems to me that inputs were not edited at all, were they? Please, rewrite this part.

(FIXED)
Section 5.3, Table 9
- How was the comparison of Ptakopet vs. GT, Bing and other web interfaces performed? Are you reporting just the feeling of annotators in using your tool with respect to their past experience with standard web MT engines? If this is the case, I do not think that the outcome is reliable and anyway you have to detail better what is the question asked to annotators.

(FIXED)
Section 5.4.1
- 2.77+-1.6 ->  2.77+-1.60 [use the same number of significant digits as in 3.85+-1.44)

(FIXED)
Section 5.4.2
- these long sentence -> these long sentences
---------------------------------------------------------------------------



============================================================================
                            REVIEWER #2
============================================================================

Detailed Comments
---------------------------------------------------------------------------
This paper could be interesting and it does propose a somewhat innovative approach to providing MT support to users with no or little knowledge of the target language. Unfortunately, it is poorly organised and rather unclear in many sections. The experiments could probably be replicated, if only the setup was better described. Whether the results would be similar would depend on the annotators. 

On the good side, there are only a couple of typos or language issues:

(FIXED)
p.1. 'translate to languages" --> "translate in" or "translate into" 
p.2. "they do not know what do they map to" -->  "they do not know what they map to"
p.8. "5-10% of inputs could not have been translated (our annotators have given up)." Does this mean they "were not translated" (i.e. could not be translated) or "they were translated" by the system even though the annotators gave up?

(FIXED)
More important are the terminology confusions introduced in Section 5.2., where it is not clear what 'first viable', 'final source/translation', 'final versions' refer to. This is exacerbated by the use of 'hypotheses' in Section 5.4.2 (are they the same as 'final versions'?). Table 8 mentions 'Source sim.' and 'Translation sim.' but the caption only adds to the confusion introduced in the text.

(FIXED)
There are irrelevant details, e.g. the plugin mentioned in p.4, but many omissions that render the text unclear and the setup hard to follow. For instance, in Section 4, it's not always clear what language(s) the input or output are supposed to be in (English, German, Czech), nor what language the annotators are expert in (p.5 "monolingual users" ???)

(FIXED)
Table 1. I assume FN is false negative, TP false positive, etc. but that should be spelled out somewhere.

(FIXED)
The evaluation of the quality estimation models in Section 3.4. is useful. I would have liked more information about the failure of QuEst++ to live up to the expectations it would perform well on unsupported language - or what those expectations were based on.

(FIXED)
What is the purpose of Table 2?

(TODO, READER'S INATTENTION)
PROSIM PROVEST: (perhaps in "no edits or deletions in already written text" could be replaced with "no edits were performed, i.e. the annotator did not change or delete anything after linearly producing the input)
Section 5.2 (p.6) refers to 'edits' but we do not know who does the editing and of what.
PROSIM PROVEST: ...extract the first input for which... -> the first version of annotator's text for which ... (ale moc se mi moje nova formulace nelibi, tak to jeste zvazte)
It also mentions "the first input the annotator expected to be successful". Does this refer to the first output of the annotator? Input to what? How do we know what the annotator 'expected'?
(the following was clearly written, IGNORE)
Section 5.4.1 then refers to "our heuristic for picking first viable inputs". That should have been described earlier (what was it?).

(TODO) -> zmenit na estimating to what extent the message would be understandable to Germans.
On the methodology side, the validation of the output (Section 5.4) by an 'annotator with a good command of German' is not adequate to assess the 'extent a native German would understand the message - you obviously need monolingual German speakers for that task.

(TODO) -> IGNORE, stejne nevim, co s tim, proste se mu to nelibilo. :-)
The evaluation survey (Section 5.3) revealed, unsurprisingly, that "the overall results are affected by the relatively low quality of the MT system".  Well, the whole purpose of the project is to help speakers with little knowledge of the target language to use MT systems, so the conclusion is obviously that you will have to take that into greater account.
---------------------------------------------------------------------------



============================================================================
                            REVIEWER #3
============================================================================

Detailed Comments
---------------------------------------------------------------------------
This paper is interesting and well written. I have no specific comment as I am not specialist of the field.
---------------------------------------------------------------------------
