@inproceedings{vishwakarma-kumar-2026-low,
    title = "Why Does Low-Rank Adaptation Work for {H}indi-{E}nglish Code-Mixing? A Geometric Analysis",
    author = "Vishwakarma, Shashank  and
      Kumar, Rakesh",
    editor = "Sarveswaran, Kengatharaiyer  and
      Vaidya, Ashwini",
    booktitle = "Proceedings of the Second workshop on Challenges in Processing {S}outh {A}sian Languages ({CH}i{PSAL}2026)",
    month = may,
    year = "2026",
    address = "Palma de Mallorca, Spain",
    publisher = "ELRA Language Resources Association (ELRA)",
    url = "https://preview.aclanthology.org/cawl-year/2026.chipsal-1.13/",
    doi = "10.63317/55yuuwkijgj9",
    pages = "127--136",
    abstract = "Low-Rank Adaptation (LoRA) enables efficient fine-tuning of large language models, yet why it works particularly well for code-mixed text remains unexplained. We propose that LoRA{'}s efficiency stems from geometric structure in multilingual pre-trained models: code-mixed embeddings concentrate in low-dimensional cross-lingual subspaces. Through spectral analysis of mBERT and MuRIL on Hindi-English (Hinglish) data, we establish that pre-trained attention weights have effective ranks of 437{--}441, while LoRA updates (r = 4,8,16) exhibit ranks of 2.1{--}5.9{---}a 136{\texttimes} average compression. Cross-lingual geometry measured via Centered Kernel Alignment shows Hinglish embeddings align strongly with Hindi (CKA=0.279) but weakly with English (0.093), compared to a monolingual baseline of 0.074. Statistical tests (Wilcoxon p {\ensuremath{<}} 10{\ensuremath{-}}19) and permutation ablations confirm these differences are robust. We interpret the convergence of geometric overlap (3.77{\texttimes} baseline) and empirical compression (136{\texttimes}) as evidence that low-rank adaptation exploits pre-existing multilingual structure. Findings are demonstrated on token-level language identification; extensions to other language pairs and tasks remain open questions."
}