# SyntaxGymFR

SyntaxGymFR is a French syntactic evaluation resource for language models. This repository contains the data and tools associated with the resource presented in our paper at **LREC-COLING 2026**.

> **Note:** A zip archive of the first stable version of the files is available in this repository (SyntaxGymFR_1.0). The experiments reported in the LREC 2026 paper are based on this archived version. The resource is still actively developing, and the files in the main repository may differ from the archived version as corrections and extensions are ongoing.

---

## Repository Structure

```
SyntaxGymFR/
│
├── data_conllu/          # Data files in CoNLL-U format
├── data_json/            # Data files in JSON format
├── data_tsv/             # Data files in TSV format
│
├── evaluation_formulas.json   # Formulas used for syntactic evaluation
├── file_status.tsv            # Status tracking for individual data files
├── results_LLMs.ods           # Results of LLM experiments
│
├── tsv2conllu.ipynb      # Notebook: convert TSV files to CoNLL-U format
├── tsv2json.ipynb        # Notebook: convert TSV files to JSON

```

---

## Data Formats

The resource is provided in three formats:

- **CoNLL-U** (`data_conllu/`): Standard format for dependency-annotated corpora, compatible with most NLP pipelines.
- **JSON** (`data_json/`): Structured format encoding syntactic hierarchy, suitable for programmatic access.
- **TSV** (`data_tsv/`): Tabular format used as the primary editing format; conversion notebooks are provided.

---

## Conversion Notebooks

Two Jupyter notebooks are provided to convert data from TSV to other formats:

- `tsv2conllu.ipynb` — Converts TSV data to CoNLL-U format.
- `tsv2json.ipynb` — Converts TSV data to JSON, generating both `ldd-hier` and `ldd-obj` representations.

---

## Evaluation

The file `evaluation_formulas.json` defines the scoring formulas used to evaluate language model performance on the syntactic test suites. Results from LLM experiments are summarized in `results_LLMs.ods`.

---

## Citation

If you use SyntaxGymFR in your research, please cite our LREC 2026 paper (details to be added upon publication).

---

## Status

This resource is under active development. Contributions, corrections, and extensions are ongoing. If you encounter any issues with the data files, please open an issue or refer to `file_status.tsv` for known corrections.
