============================================================================ 
CODI 2024 Reviews for Submission #14
============================================================================ 

Title: Feature-augmented model for multilingual discourse relation classification
Authors: Eleni Metheniti, Chloé Braud and Philippe Muller


============================================================================
                            REVIEWER #1
============================================================================

---------------------------------------------------------------------------
Reviewer's Scores
---------------------------------------------------------------------------
                   Appropriateness (1-5): 5
      Originality / Innovativeness (1-5): 3
           Soundness / Correctness (1-5): 4
             Meaningful Comparison (1-5): 4
                    Recommendation (1-5): 4
               Reviewer Confidence (1-5): 3

Detailed Comments
---------------------------------------------------------------------------
The authors train transformer-based multi-lingual language models for discourse relation classification, across discourse frameworks and languages. 
They extend a previous (monolingual) feature-based approach and analyse it for a multilingual setting. They use the DISRPT shared task datasets from 2021 (i.e. RST/PDTB/SDRT). They improve the SOTA. 
The authors adopt a lot of previously proposed work (e.g. the best system from the 2021 shared task, label merging in Sec 3.3), which whilst obviously is not novel, shows that they are aware of previous work and the novel ideas that they do present in the paper build on these nicely.
Overall I think the paper is a good contribution, that could be improved with some rewordings (see detailed comments below).
A nice addition would be some more detailed error analysis: are there particular relations that the system struggles with, regardless of language?


Detailed comments:

l262: quotation marks not consistent
l297 Section heading: would choose either novel OR additional for the heading

l318: feature embedding as tokens: how exactly was this done? i.e. if the feature is nuc_children, is simply a "2" concatenated to the input or the something that specifies the name of the feature too, like "nuc_children: 2"? would be good to have more information

l344 and l362 headings are capitalised, whilst the others aren't

l352 a "do" is missing: "or do not cover all relations"

l394 would delete "and" and "the" --> "languages, with masked language modeling ....."

l423 would say "second row" instead of "second line"

In Tab. 2 I would change the second row to make it more clear by saying "Additional tokens" for DisCoDisCo or "Switching units" for DiscReT

l452 this sentence is very vague: "overall features are beneficial"? could this be reworded to make it more informative?

l459 how is "successful" defined? would maybe use a more precise term, like accurate? (in terms of F1 score, or something similar)

l480 would be useful to have the languages for these datasets as readers may not be familiar with the way the datasets were named (and also in lines 510/11 etc. too)

l491 no comma
---------------------------------------------------------------------------



============================================================================
                            REVIEWER #2
============================================================================

---------------------------------------------------------------------------
Reviewer's Scores
---------------------------------------------------------------------------
                   Appropriateness (1-5): 5
      Originality / Innovativeness (1-5): 3
           Soundness / Correctness (1-5): 4
             Meaningful Comparison (1-5): 5
                    Recommendation (1-5): 4
               Reviewer Confidence (1-5): 3

Detailed Comments
---------------------------------------------------------------------------
The paper explores a multilingual approach to classification of explicit and implicit discourse relations using transformer-based models (XLM-RoBERTa, mBERT and DistilmBERT). Reprising the DISRPT 2021 shared task, the authors evaluate its dataset, compare previous systems and enhance features to achieve state-of-the-art performance.

Strong point of this paper is exploration of impact of various features on model performance together with the effect of annotating relation direction.

Although the paper is in line with current trends and does everything effectively, I would be really happy if the authors commented on how various properties of the subcorpora influenced the results, e.g. taking into the variability in sizes and frameworks. The analysis of errors is also limited.

What I also would like to see is some information on the computational costs of the solution which seems to be more and more important in this resource-intensive world.

The paper looks like an exercise before DISRPT 2025 shared task: after the 2021 version the authors plan to use the 2023 benchmark. Would be great, see you in 2025!


Style remarks:

â€“ for Footnotes 2â€“5: please remove space between the footnote mark and the preceding word (to keep it glued just like it is for Footnote 1 in line 201)

â€“ in bibliography: please replace arXiv papers which were published in journals/conference proceedings with their 'proper' entires (e.g. https://arxiv.org/pdf/1910.03771.pdf is now https://aclanthology.org/2020.emnlp-demos.6.pdf)
---------------------------------------------------------------------------


-- 
CODI 2024 - https://softconf.com/eacl2024/CODI-2024 