Hi-SEMFLOW: Lie Algebra–Based Semantic Flow for Span-Level Informal Language Identification in Hindi
Manikandan Ravikiran, Tanmay Tiwari, Vibhu Gupta, Rohit Saluja
Abstract
Informal Hindi text frequently contains multi-token slang and idiomatic expressions whose correct identification requires consistent span boundaries. Transformer-based token classifiers, despite strong contextual representations, often produce fragmented or structurally invalid BIO sequences due to largely local predictions. We propose Hi-SEMFLOW, a Lie algebra–based semantic flow framework that models span consistency as a continuous refinement process over label logits. Instead of discrete structured decoding (e.g., CRFs), Hi-SEMFLOW learns context-dependent transition operators derived from antisymmetric generators and propagates structural information through smooth, fully differentiable transformations. This formulation integrates structural bias directly into end-to-end training without requiring dynamic programming or hard decoding constraints. Experiments on the HiSlang-4.9k benchmark show that Hi-SEMFLOW improves span-level F1 by up to 2–3 absolute points and yields consistent macro-F1 gains across Hindi-pretrained encoders. Extensive ablations demonstrate that continuous geometric refinement provides a flexible and effective alternative to discrete structured decoding for span-centric sequence labeling.- Anthology ID:
- 2026.chipsal-1.14
- Volume:
- Proceedings of the Second workshop on Challenges in Processing South Asian Languages (CHiPSAL2026)
- Month:
- May
- Year:
- 2026
- Address:
- Palma de Mallorca, Spain
- Editors:
- Kengatharaiyer Sarveswaran, Ashwini Vaidya
- Venues:
- CHiPSAL | WS
- SIG:
- Publisher:
- ELRA Language Resources Association (ELRA)
- Note:
- Pages:
- 137–153
- Language:
- External URL:
- https://lrec.elra.info/lrec2026-ws-chipsal-14
- DOI:
- 10.63317/2duxp4m72ddj
- Cite (ACL):
- Manikandan Ravikiran, Tanmay Tiwari, Vibhu Gupta, and Rohit Saluja. 2026. Hi-SEMFLOW: Lie Algebra–Based Semantic Flow for Span-Level Informal Language Identification in Hindi. In Proceedings of the Second workshop on Challenges in Processing South Asian Languages (CHiPSAL2026), pages 137–153, Palma de Mallorca, Spain. ELRA Language Resources Association (ELRA).
- Cite (Informal):
- Hi-SEMFLOW: Lie Algebra–Based Semantic Flow for Span-Level Informal Language Identification in Hindi (Ravikiran et al., CHiPSAL 2026)