A Switch-Point-Aware Contrastive Approach to Sentiment Analysis of Hinglish Code-Mixed Text

Abstract

With the increasing use of social media in non-English-speaking regions, especially in India, people often use Romanized Hindi and English together in their online communication. In a single sentence, they frequently mix Romanized Hindi and English, creating code-mixed text. However, most multilingual transformer models are pre-trained primarily on monolingual data. As a result, NLP systems face challenges when processing code-mixed text, as a single word may be fragmented into meaningless subword pieces, making it difficult for the model to capture its semantic meaning accurately. In this dissertation, we propose a parameter efficient neural architecture consisting of three main components to address these challenges: First, there is a character-level CNN encoder, which handles spelling differences such as "nahi", "nahin", "nah", and "nai" through the chracter n-gram pattern. Next, there is a frozen XLM-R backbone(Conneau et al., 2019) , the top three layers, which are partly fine-tuned at a slower rate by which it provides rich cross lingual embeddings. Finally, there is a switch-point-aware bilingual gate that spots where the language label switches and blends two adapters using a learned gate weight.During training, it uses Supervised Contrastive Loss to learn better feature representations and Cross-Entropy Loss for classification. Since human annotators agreed on labels only 55% of the time, we use label smoothing to reflect this uncertainty and prevent the model from becoming overly confident in noisy labels. Evaluated on the SentiMix 2020 benchmark(Patwa et al., 2020), our proposed architecture achieves a weighted F1 score of 0.705, which outperforms the baseline model M-BERT (0.654 F1) and is comparable to fully fine-tuned transformer models while requiring only one-tenth of the trainable parameters.Adapter gate visualizations provide interpretable evidence that the gating mechanism captures linguistically meaningful codemixing structure. The architecture is designed to generalize to other code-mixed language pairs through its modular adapter design.