A comparative machine translation study has demonstrated that bidirectional pre-training paired with fine-tuned Transformer architectures achieves the highest translation accuracy for the low-resource English–Assamese language pair. Computational linguists evaluated how word representations, recurrent neural network frameworks, pre-trained sequence-to-sequence Transformers, and instruction-tuned large language models handle the complex grammatical demands of Assamese. Because Assamese possesses an intricate morphological system characterized by widespread inflectional variation, compounding, and unique phonetics, conventional word-for-word translation models routinely encounter severe out-of-vocabulary bottlenecks. The experimental investigation established that combining subword tokenization with an auxiliary bidirectional training phase in a multilingual denoising autoencoder reached bilingual evaluation understudy scores of 28.93 for Assamese-to-English translation and 28.57 for English-to-Assamese translation. These quantitative outcomes, supported by human assessments of fluency and semantic adequacy, underscore the ongoing necessity of specialized, domain-tailored neural architectures over generic zero-shot language models when translating digitally under-resourced languages.
Developing reliable neural machine translation systems for Assamese has long remained hindered by the acute scarcity of structured parallel training resources. In traditional phrase-based statistical machine translation, systems frequently fail to preserve long-distance sentence dependencies, subtle contextual relationships, and complex clause structures. Modern neural sequence-to-sequence frameworks improve translation by projecting vocabularies into dense continuous vector spaces where syntactic and semantic affinities are mathematically preserved. However, when a language features rich nominal and verbal inflections like Assamese, standard vector modeling often isolates related word forms into disconnected tokens. To address this technical hurdle, the researchers compiled and systematically evaluated a dedicated parallel training corpus comprising 175,000 sentence pairs across four diverse domains, providing a rigorous testbed to benchmark embedding algorithms and modern translation architectures.
The underlying training data was curated from four distinct textual genres to expose the translation models to varied linguistic styles, cultural idioms, and specialized vocabularies. The sports segment incorporated fast-paced match commentaries and event reports dense with action verbs and domain-specific terminology. The literary collection integrated poetry and short stories characterized by figurative rhetoric, rare vocabulary, and complex syntactic compounding. The tourism domain supplied cultural descriptions of regional heritage, featuring location names such as the Kamakhya Temple and indigenous cultural events like the Bihu festival. Finally, the news component gathered editorial reporting from Asomiya Pratidin to capture formal contemporary prose. Sentence alignments across the bilingual corpus were produced using a hybrid computational approach integrating IBM Model 1 probabilities with Hunalign length-based heuristics. Manual validation by native Assamese linguists and professional translators resolved idiom paraphrasing, dialectal variations, and syntactic anomalies, yielding an alignment error rate of 7.7 percent on a manually annotated benchmark.
Data pre-processing involved rigorous cleaning and tokenization strategies designed to maximize vocabulary coverage while eliminating extraneous linguistic noise. English sentences were tokenized utilizing the Moses toolkit, while Assamese text was segmented using Byte-Pair Encoding through SentencePiece with 10,000 merge operations. This subword decomposition substantially compressed the active vocabulary, reducing English from 45,000 words down to 12,000 subwords and Assamese from 62,000 words to 15,000 subword units. Heuristic sentence boundary detection addressed script-specific markers, employing rule-based identification of the Assamese daa symbol alongside standard English punctuation, achieving 98.7 percent segmentation accuracy on human-annotated test data. Text cleaning pipelines applied Unicode NFKC normalization, removed web markup and uninformative punctuation, and filtered stopwords via curated Indic NLP and NLTK dictionaries. Furthermore, tokens appearing fewer than five times were replaced with dedicated unknown markers, eliminating lexical sparsity and reducing overall vocabulary size by 35 percent in Assamese and 28 percent in English without degrading translation quality.
The investigation systematically analyzed three prominent continuous word embedding techniques within recurrent neural network baselines: Word2Vec, GloVe, and FastText. In recurrent sequence-to-sequence networks equipped with Bahdanau additive attention, 300-dimensional input embeddings serve as the foundation for the encoder and decoder hidden states. Word2Vec structures word vectors using local contextual windows via Skip-gram or Continuous Bag of Words objectives, optimizing semantic proximity between co-occurring terms. GloVe incorporates global co-occurrence matrix factorization across the entire training corpus to balance local context with corpus-wide statistical distributions. Both architectures, however, treat each word as an indivisible atomic token, which causes them to struggle when encountering unobserved inflections or rare compound derivatives. FastText overcomes this limitation by representing individual words as bags of character n-grams. By summing subword component vectors, FastText reconstructs continuous vector representations for unseen lexical items, providing critical resilience when processing morphologically sophisticated languages like Assamese.
Empirical comparisons between Gated Recurrent Units and Long Short-Term Memory networks demonstrated distinct operational differences across varying training data volumes. Both recurrent models were evaluated across incremental data slices scaling from 10,000 sentence pairs up to the full 175,000-sentence dataset. Across all corpus increments, translation quality improved steadily, marked by rising BLEU and METEOR metrics alongside consistent declines in Translation Edit Rate scores. The Gated Recurrent Unit architecture consistently surpassed the Long Short-Term Memory network at every evaluated corpus scale. With FastText embeddings on the complete dataset, the GRU achieved 25.95 BLEU points in the English-to-Assamese direction and 24.99 BLEU points in the Assamese-to-English direction. The researchers attributed this persistent margin to the simpler gating mechanism of the GRU, which contains fewer trainable parameters than the multi-gate memory cell of the LSTM, enabling quicker parameter convergence and substantially reducing overfitting risks under constrained data conditions.
The comparative embedding experiments confirmed that subword-level modeling provides clear advantages over whole-word continuous vector formulations in low-resource environments. On the full parallel corpus, FastText delivered the highest automatic evaluation scores for both recurrent architectures, generating a GRU score of 25.95 BLEU in the English-to-Assamese direction compared to 23.00 BLEU achieved by Word2Vec. Principal component analysis visual inspection of two-dimensional vector projections further corroborated these quantitative metrics. While Word2Vec and GloVe produced loosely dispersed clusters with considerable overlapping boundaries between distinct grammatical categories, FastText formed tightly grouped, semantically coherent vector clusters. Although FastText occasionally exhibited slightly elevated Translation Edit Rate values due to minor surface form mismatches caused by granular subword segmentation, its superior scores in human adequacy and fluency ratings confirmed that character n-gram composition preserved underlying semantic meaning far more effectively than whole-word embedding algorithms.
Transitioning from recurrent baselines to self-attention Transformer architectures yielded substantial performance gains across both translation directions. The researchers evaluated MarianMT alongside mBART, a multilingual sequence-to-sequence model pre-trained on diverse languages with Indic script representation. Unlike the recurrent frameworks that relied on external static embeddings, the Transformer models leveraged pre-trained subword representations fine-tuned using the Adam optimizer with gradient clipping. MarianMT reached 27.65 BLEU for Assamese-to-English and 27.12 BLEU for English-to-Assamese, while baseline mBART achieved 28.12 BLEU and 27.89 BLEU respectively. Multi-head self-attention mechanisms enabled these architectures to compute direct contextual relationships across all tokens simultaneously, bypassing the sequential information bottleneck inherent to recurrent hidden state transitions. This architectural capability proved decisive for Assamese, where long-distance clause reordering and extensive syntactic dependencies frequently degrade the translation coherence of recurrent networks.
To maximize cross-lingual knowledge transfer, the authors implemented an auxiliary Bidirectional Training pre-training phase that optimized mBART on combined translation trajectories. In this configuration, English-to-Assamese and Assamese-to-English sentence pairs were merged into a single training set, with directional prefix tokens designating the intended translation target. The unified model underwent joint pre-training for three epochs at a reduced learning rate of 0.00003 prior to independent direction-specific fine-tuning. This bidirectional parameter sharing allowed the encoder and decoder layers to align shared subword representations across both translation directions. The resulting mBART+BiT variant achieved the highest quantitative performance of the entire study, recording 28.93 BLEU for Assamese-to-English and 28.57 BLEU for English-to-Assamese. This cross-directional optimization provided an absolute improvement of 0.5 to 0.7 BLEU points over standard mBART, while simultaneously lowering Translation Edit Rates, demonstrating enhanced structural alignment and syntactic symmetry.
A detailed word-frequency distribution analysis revealed how architectural choices influence translation quality across different lexical scarcity bands. Following established experimental protocols, the evaluation vocabulary was partitioned into high-frequency items appearing at least 100 times, medium-frequency terms occurring between 10 and 99 times, and low-frequency words appearing fewer than 10 times. Translation quality degraded across all models in the lowest frequency tier, illustrating the pervasive long-tail distribution challenge in natural language processing. However, while the recurrent GRU configuration experienced a steep drop of 7 to 8 BLEU points when transitioning from high to low frequency tokens, mBART maintained far greater resilience, dropping by only 4 to 5 BLEU points. Furthermore, the bidirectional pre-trained mBART+BiT model generated an additional 0.5 BLEU gain specifically within the rarest lexical band, proving that joint cross-directional exposure strengthens the representation of infrequent words and complex inflected roots.
Qualitative evaluations and comparative benchmarking against an instruction-tuned zero-shot large language model highlighted the practical boundaries of generalist versus specialized translation systems. On a 100-sentence FLORES-200 benchmark test, zero-shot GPT prompting produced competitive output when translating Assamese into English but underperformed dramatically on English-to-Assamese generation. The zero-shot model frequently produced overly literal word-for-word renderings and grammatically disjointed Assamese inflections. Qualitative error auditing revealed that while recurrent models suffered from lexical substitutions like rendering stadium as field and cultural mismatches like translating Bihu festival into generic songs and dance, Transformer models consistently preserved regional named entities, cultural concepts, and complex grammatical clauses, confirming the vital importance of specialized pre-training.
Subject of Research: Neural machine translation optimization for bidirectional English and Assamese language pairs
Article Title: Improving Bidirectional English–Assamese Neural Machine Translation with Word Embeddings and Transformer Models
Article References: Nath, B., Pandey, K. K., Yadav, V., & Dixit, P. (2026). Improving Bidirectional English–Assamese Neural Machine Translation with Word Embeddings and Transformer Models. International Journal of Networked and Distributed Computing, 14(1), Article 17. https://doi.org/10.1007/s44227-025-00080-1
Image Credits: AI Generated
DOI: 10.1007/s44227-025-00080-1
Keywords: neural machine translation, Assamese language, bidirectional pre-training, Transformer architectures, mBART, MarianMT, FastText embeddings, Word2Vec, GloVe, Byte-Pair Encoding, Gated Recurrent Unit, Long Short-Term Memory, BLEU evaluation, subword segmentation, low-resource translation, Bahdanau attention, Translation Edit Rate, parallel corpus alignment
News Source: Denise Maddox. (November 19, 2026). Bidirectional Transformer Pre-Training Boosts Low-Resource English and Assamese Machine Translation. Scienmag.



